National Center for Environmental Health Radiation Studies Branch Radiation Studies Branch | NCEH | Contact Us

March 12, 1999

The attached is a preliminary evaluation of HTDS power calculations by Mr. Eduard Hofer of Germany. This material demonstrates that statistical power can be substantially lower than stated by the authors of HTDS. The primary factors that would influence low power in HTDS are:

1. Effectively ignoring the effects of uncertainty and correlations by fixing individual dose estimates at mean or median values,
2. Not accounting for the existence of weak to moderate correlations among the individual dose estimates,
3. Overestimation of the true variance of doses in the cohort by treating uncertainty as a Berkson measurement error,
4. Over-estimating the true mean dose for the cohort (given that uncertainty in many parameters are partially shared among members of the cohort),
5. The recognition that computed power is an uncertain quantity and therefore it should be represented as a subjective probability distribution; such distributions indicate the chance for power to be much lower than specified in HTDS,
6. Selection of a target risk level when computing statistical power that is insufficiently low to ensure that the design of HTDS is adequate for the detection of true dose-response relationships. This is especially a problem when the true dose response is near the lower part of (or slightly below) the range of the dose response relationship reported from other studies, and
7. Underestimation of the increase in the background incidence rate due to intensive medical screening of the cohort.

F. Owen Hoffman, Ph.D.

President and Director

________________________________________________________________________________________

Eduard Hofer St. Sebastian 5, 84405 Dorfen, Germany
March 1999

POWER CALCULATIONS, AND ESTIMATION OF PARAMETER VALUES OF THE RELATIONSHIP BETWEEN DOSE AND DISEASE PROBABILITY

1.0 Consistency Among Individuals' Dose Estimates is Lost When Each Individual's Dose is Represented by the Median Estimate

The HEDR study acknowledges that parameters used in its dose reconstruction process are uncertain. Some of these uncertainties (those associated with release, dispersion, deposition and uptake modeling, etc.) are common to many individuals' dose estimates, while others (associated with food consumption and life style modeling etc.) are individual-specific.

HEDR expresses parameter uncertainty by subjective probability distributions. These distributions quantify the state of knowledge as judged by the HEDR analysts. The propagation of uncertainty through the HEDR models results in a subjective probability distribution for each individual's dose estimate. Unfortunately, these resulting distributions cannot be obtained analytically, due to the complexity of the HEDR models. However, a random sample is produced numerically and serves as an approximate quantitative expression of the combined influence of the parameter uncertainties on the estimation of the individuals’ dose values.

This random sample (the 100 alternative realizations of the HEDR dose) was drawn such that the correlation among individuals' dose estimates (due to common contributors to uncertainty) is preserved. The k-th of the 100 realizations, k=1,...,100, taken for each individual, produces a consistent vector of dose estimates as they are obtained using the same value for each of the uncertain parameters that are common to many individuals' dose estimates.

To obtain a median dose value for each member of the cohort, HTDS orders the 100 realizations of dose (by increasing value of the dose estimate) for each individual and forms a dose vector for the cohort that is made up of the 50th (or 51st) of the ordered dose estimates per individual. By this ordering process, the consistency is lost. This dose vector may contain a dose estimate for individual x_i at location y_i that is obtained with a high value for an uncertain dispersion parameter, together with a dose estimate for individual x_j at location y_j that is obtained with a low value for the same parameter. The same applies for wind direction, deposition velocities, uptake, etc.

2.0 An Evaluation of HTDS Values of Statistical Power

Computation of statistical power is affected by uncertainty in dose estimates, common contributors to uncertainty in individuals' dose estimates, the value of the targeted excess risk, systematic over- or underestimation of the dose and/or of the value used for the average background probability of disease. The effect of each of these factors is illustrated as follows:

2.1 Methods and assumptions

The empirical distribution of the 3190 individual median dose estimates of HTDS is approximated by a lognormal distribution with the same mean value and standard deviation as in HTDS. It has the parameters

= 4.7349 = 0.9687

which corresponds to

mean value = 182 mGy std. dev. = 227 mGy.

This lognormal distribution of individual doses has a median value of 114 mGy (HTDS: 104 mGy) and a 99.2% fractile of 1170 mGy (HTDS: 1000 mGy, cf. Chapter VIII.B, page 11 of 96 of the final draft).

The frequency distribution of individual dose is truncated at 2900 mGy (maximum of the 3190 median values of HTDS is 2842 mGy).

A normal distribution is assumed for the 3190 ratios of the 95% fractile to the median value of the 100 realizations of individual doses. The information given in the HTDS report (cf. Chapter VIII.B, page 10 of 96 of the final draft) suggests a mean value of this ratio of 3.8 , a standard deviation of this ratio of 0.9363 and truncations at 1.5 and 12.0. The 100 realizations of each individual's dose are assumed to be well approximated by a lognormal distribution (see HTDS Chapter VIII.B, page 8 of 96).

With respect to "benign nodules", the HTDS estimates for the average background probability of disease are used (males: 0.05 ± 10% and females: 0.1 ± 10%, cf. HTDS, Table VIII-47, page 38 of 96), and their uncertainty estimates are interpreted as one standard deviation of a normal distribution. For "thyroid cancer" 0.003 ± 20% for males and 0.007 ± 20% for females is used. These values are taken from Chapter V, page 10 of 142 of the HTDS draft final. The ± 20% uncertainty is based on expert judgement and is interpreted as ± 1 standard deviation of a normal distribution. Furthermore, an equal proportion of males and females in the cohort is assumed.

The power is computed for detection of the targeted excess absolute increase (0.05 per person for "benign nodules" and 0.025 per person for "thyroid cancer") in probability for disease at a dose of 1 Gy, using a significance level of 5%. An excess absolute risk of 0.05 can be interpreted as "on average 5 cases in excess to background, among 100 individuals exposed to 1 Gy".

A sample of 3190 dose values is drawn at random and disease cases are generated randomly according to the linear model (formula [1] in Chapter VII, page 10 of 18, HTDS draft final) with background value as mentioned above. The parameter value or slope of the dose response is chosen such, that it corresponds to the targeted excess probability for disease at 1000 mGy.

2.2 Statistical power should be expressed as a subjective probability distribution

The statistical "power" is the probability to reject the hypothesis that there is no relationship between dose and disease probability, in favor of the alternative hypothesis, namely that there is a linear relationship with excess probability as targeted, if indeed the alternative is true. The power is obtained for the test statistic given in Appendix H (of HTDS, June 1994), without stratification, and via Monte Carlo simulation. To quantify the influence of uncertainty on the computed power, a separate simulation is performed to produce a subjective probability distribution for "power."

A subjective probability distribution is obtained for power because the value for the average background probability of disease is uncertain and because there is uncertainty as to which set of 3190 dose values, drawn at random from the (joint) uncertainty distribution of the individual dose estimates, is the appropriate one.

2.3 How sensitive is the computed power to the targeted excess probability for disease?

Two cases are investigated:

The results obtained with median values of the dose estimates (uncertainties neglected) show a mean value for power that is only 52.7% when the target excess absolute risk is 0.05 (person Gy)^-1 for benign nodules. For a target excess absolute risk of 0.01 (person Gy)^-1 the power drops to less than 10% (Table 1). For thyroid cancer, the target risk of 0.025 (person Gy)^-1 produces a mean power of less than 80%, but at a risk of 0.01 (person Gy)^-1, the mean power drops to 33.5% (Table 2). Thus, for a target risk of 0.01 (person Gy)^-1, and using median dose values, HTDS does not seem to have sufficient statistical power to safely detect the targeted dose response.

2.4 What does uncertainty in dose estimates do to the computed power?

Three cases are investigated with the target values for excess absolute risk per person as in Case 1:

Case 3

The uncertainty in the individuals' dose estimates is taken into account but the correlation effect from common contributors to uncertainty is neglected.

Cases 4 and 5

The uncertainty in the individuals' dose estimates is taken into account together with a rank correlation among the individuals' dose estimates of approximately 0.25 (case 4) and 0.5 (case 5) to simulate the effect of common contributors to uncertainty.

Clearly, the effect of correlation due to common contributors to uncertainty is noticeable as comparison of Case 3 to Cases 4 and 5 shows in either Table 1 (benign nodules) or Table 2 (thyroid cancer). A rank correlation of 0.25, although a relatively low correlation, may be too high for some pairs of individuals in the cohort while simultaneously too low for others. An exact comparison will require use of the 100 consistent vectors of dose estimates generated for the 3190 individuals in HTDS.

If we compare cases 1 and 3 (or 1 and 4) in Tables 1 and 2, respectively, we find that the uncertainties given by HEDR for the reconstructed doses had the effect of increasing the computed power. This increase is only meaningful if true doses can be considered as occurring randomly from the lognormal subjective probability distributions of dose specified for each individual by the 100 alternative realizations of the dose estimate. A situation like this applies if the median is a measured value and the individual lognormal distributions quantify measurement error (Berkson measurement error). As a consequence of this error, the variance of the true values is always larger than that of the measured values and therefore the computed power is larger if measurement error is accounted for. It should, however, be realized that the case of reconstructed doses and their uncertainty is

Table 1 Fractiles of the subjective probability distribution of power for the health effect "benign nodules"

a Targeted Excess Absolute Risk (person Gy)^-1.

b Uncertainty in dose estimates is ignored.

c Dose estimates with uncertainty; however, true doses were assumed lower by a factor of 1/3.

d Dose estimates with uncertainty; however, true background was assumed higher by a factor of 3.

e Subjective probability distribution representing uncertainty in computed power.

Table 2 Fractiles of the subjective probability distribution of power for the health effect "thyroid cancer."

a Targeted Excess Absolute Risk (person Gy)^-1.

b Uncertainty in dose estimates is ignored.

c Dose estimates with uncertainty; however, true doses were assumed lower by a factor of 1/3.

d Dose estimates with uncertainty; however, true background was assumed higher by a factor of 2.

e Subjective probability distribution representing uncertainty in computed power.

conceptually different from that of Berksonian measurement error. The true dose values may have larger or smaller variance than the medians – it is unknown. The computed increase in power is the result of an idealization. This idealization lies in the assumption that the true dose values vary about the medians of the 100 reconstructed dose values according to the subjective lognormal distributions that quantify uncertainty.

2.5 How sensitive is the computed power to overestimation of dose by a factor of three?

Two cases are investigated:

Dose estimates are as in Case 4 (uncertainties taken into account together with a rank correlation of approximately 0.25) and it is assumed that true dose values are lower by a factor of 1/3.

In this case the mean computed power is only 30.4% for benign thyroid nodules (Table 1) and 49% for thyroid cancer (Table 2).

Clearly, if true dose values are systematically less than those estimated, because HEDR either overestimated doses (or was too pessimistic in the quantification of uncertainties), the computed power will be diminished substantially, as the comparison of Case 4 to Case 6 shows in Tables 1 and 2, respectively.

2.6 How sensitive is the computed power to underestimation of the average background probability of disease?

To study this effect, the following two cases are investigated, using dose values as in case 4 (uncertainty taken into account together with a rank correlation of approximately 0.25) and increasing the value for average background probability of disease by a factor of three (for "benign nodules") and a factor of two for "thyroid cancer":

The results in Table 1 for benign nodules show a mean power of 65.3% and 7.9%, respectively, for Cases 8 and 9. The results in Table 2 for thyroid cancer show a mean power of 85% and 45.8%, respectively, for Cases 8 and 9. In these cases, increasing the background incidence of thyroid disease decreases the computed power.

2.7 Maximum likelihood estimates are only one possible set of parameter values of the risk model. In fact, they may often be a next to arbitrary choice from a set of possibly applicable parameter values

The maximum likelihood estimates of parameters of the risk model are only one of many possible sets of parameter values. The maximum likelihood estimate is that value for which the likelihood function of the observations assumes its maximum. However, often the likelihood function does not have a very pronounced maximum or is even nearly flat over a wide range of possible parameter values. Under these circumstances, the maximum likelihood estimate is a rather arbitrary choice from a wide range of possible parameter values. Even if the likelihood function exhibits a clear maximum, the estimate is still only telling half the story since nothing is said about the range of possible values and how likely they are, given the evidence.

Whenever the number of cases is small, and uncertainty is involved, Bayesian methods are widely applied.

If further analyses were to be performed on the HTDS cohort, a Bayesian approach is recommended. The immediate advantage of a Bayesian approach is that a subjective probability distribution is obtained for the slope of the dose response model. This distribution

• quantifies the uncertainty in the dose response due to limited sample size and number of cases as well as uncertainty in dose estimates (inclusive correlation effects). The 100 alternative realizations of individuals' dose estimates provided by HEDR can be immediately incorporated into the Bayesian approach.

• also gives the subjective probability for the dose response parameter to exceed a given limiting value.

• provides a 90% uncertainty range or upper (resp. lower) 95% uncertainty limits of the parameters.

The output from a Bayesian approach gives the spectrum of possible parameter values but also indicates how likely they are in the light of the available evidence.

Radiation Studies Branch | NCEH | Contact Us

CDC Home | Search | Health Topics A-Z

This page last reviewed January 6, 2000

Centers for Disease Control and Prevention
National Center for Environmental Health
Radiation Studies Branch