WISQARS™ Cost of Injury Reports Help Menu
4.5 Reliability (Stability) of Cost-of-Injury Estimates
Cost Estimates Based on System-Provided Data Only
Effective in 2012, a cost estimate that is based exclusively on system-provided data is evaluated for statistical reliability (stability) by considering two basic sources of variation:
- the count (or estimated count) of cases associated with the cost estimate;
- the variation in assigned unit costs (medical and/or work loss) across the case records used for the cost estimate.
Prior to this date, stability assessments for such cost estimates referred only to case counts or estimated case counts. Under the present approach, such estimates may be annotated as statistically unreliable (unstable) due to small contributing case counts, excessive case-to-case variation in assigned unit costs, or the combined effects of both.
For fatal injuries, any cost estimate based on 20 or fewer case records is considered statistically unreliable. Even when based on more than 20 case records, however, a cost estimate for fatal injuries can be statistically unreliable due to high variability of the case-level costs. The latter determination is based on the estimated coefficient of variation (CV) for the cost estimate, which measures its variability relative to its magnitude. If the estimated CV exceeds the threshold value of 0.22, the cost estimate is considered unreliable no matter how large the contributing case count.
For nonfatal injuries (i.e., hospitalized or ED treated-and-released), any cost estimate based on 20 or fewer surveillance records or an estimated national case count of 1200 or less is considered statistically unreliable. Even when based on a sufficiently large case count (surveillance / national estimate) a cost estimate for nonfatal injuries can be statistically unreliable due to high variability of the case-level costs. The latter determination is based on the estimated CV for the cost estimate. If the estimated CV exceeds the threshold value of 0.30, the cost estimate is considered unreliable regardless of the number of contributing surveillance records or the estimated national case count.
Cost Estimates That Combine System-Provided Data and User-Provided Data
The general process for calculating cost estimates using a combination of system-provided data and user-provided data is described in Section 4.1. The first step in that process involves the calculation of preliminary average cost estimates at the lowest level of stratification indicated in the user interface. Each preliminary average cost estimate will be based entirely on system-provided data and as such will inherit the two sources of variation described above. The next step involves the multiplication of each preliminary average cost estimate by a corresponding user-entered case count to arrive at a total cost estimate for the indicated category (stratum). Each resulting elementary-level total cost estimate is subject to three distinct sources of variation:
- the count (or estimated count) of cases associated with the preliminary average cost estimate;
- the variation in assigned unit costs (medical and/or work loss) across the case records used for the preliminary average cost estimate;
- the user-provided case count that is multiplied by the preliminary average cost estimate.
An elementary-level total cost estimate may be statistically unreliable due to any one of these sources of variation or due to their combined effects. When a preliminary average cost estimate is deemed unreliable, the corresponding elementary-level total cost estimate will also be treated as unreliable regardless of the user-provided case count. Even when a preliminary average cost estimate is reliable, the corresponding elementary-level total cost estimate may nonetheless be unreliable if the user-provided case count is small. The latter determination is based the estimated CV for the elementary-level total cost estimate, which is compared to the appropriate threshold value (see the threshold values specified above for system-generated cost estimates associated with fatal and nonfatal injuries).
To obtain total cost estimates for a higher-level stratum (if any) all of the elementary-level total cost estimates belonging to that stratum are summed. It is entirely possible that most or all of the elementary-level total cost estimates falling under a higher-level stratum are separately unreliable, yet collectively result in a total cost estimate for the higher-level stratum that is deemed reliable. This determination is made by calculating the estimated CV for the higher-level total cost estimate, and comparing it to the appropriate threshold value (see above). Corresponding average cost estimates are not calculated for higher-level strata.
Notes: In the present context, a lack of statistical reliability (stability) for a given cost estimate should be interpreted as meaning that the estimate would be likely to fluctuate substantially from one hypothetical base year (not index year) to the next. It does not mean that the estimate is biased or that it is based on inaccurate or unreliable data.
Statistically unreliable estimates are annotated using several distinct symbols to identify the reason(s) for the lack of reliability. These notations may be helpful in determining whether and how reliable (stable) estimates might be obtained, such as by collapsing reporting categories (strata).