Hypothesis Testing when using the NHANES-CMS linked data

The t-test is used to test the null hypothesis that two population means or proportions, θ₁ and θ_2, are equal OR, equivalently, that the difference between two population means or proportions is zero. To test this hypothesis, assuming the covariance is small, as is the case with NHANES data, the following formula is used

Equation for t-Test Where Covariance is Small

(1)
where,

₁ is an estimate of θ₁ based on a probability sample,

₁ is an estimate of the standard error of  ₁,

₂ is an estimate of θ_2,

and ₂ is an estimate of the standard error of  ₂.

In instances where the t statistic is based on a small number of independent pieces of information (i.e. a small number of degrees of freedom [<30]), the statistic given in equation 1 follows a Student’s t distribution with mean=0 and unit variance with n degrees of freedom. In the NHANES 1999-2002 sample, the degrees of freedom depend on the number of first stage units, or PSUs, containing observations and is defined as the number of PSUs minus the number of strata. (See Sample Design module for more information.)

The equality of means is usually tested at the .05 level of significance.

References:

Cochran, WG. Sampling Techniques. John Wiley & Sons. 1977.

Lohr SL. Sampling: Design and Analysis. Duxbury Press. Pacific Grove 1999.

In this task, you will use SUDAAN to calculate a t-statistic and assess whether the mean systolic blood pressures (SBP) in males and females age 20 years and older are statistically different.

Step 1: Set Up SUDAAN to Produce Means

Follow the steps in the summary table below to produce the mean SBP using the SUDAAN procedure proc descript.

Important Note: These programs use variable formats listed in the Tutorial Formats page. You may need to format the variables in your dataset the same way to reproduce results presented in the tutorial.

Step 2: Review SUDAAN Means Output

• 9,056 respondents had information on systolic blood pressure (SBP).
• The results indicate the mean SBP was 124 for males and 122 for females.

Step 3: Perform t-test to Test for Significance

A t-test is used to test whether the mean SBP between males and females obtained in the previous step is statistically significant different.

Request the t-test from the SUDAAN procedure proc descript and follow the steps in the summary table below.

Important Note: Note that this program and the previous program to produce means in Step 1 are identical up to the var statement.

Important Note: These programs use variable formats listed in the Tutorial Formats page. You may need to format the variables in your dataset the same way to reproduce results presented in the tutorial.

Step 4: Review SUDAAN t-test Output

• 9,056 respondents had information on systolic blood pressure where the degrees of freedom was 29.
• To test the hypothesis that the difference between the two means is zero, the t-statistic with 29 degrees of freedom is computed as 2.64. The p-value is 0.0132, which indicates that the probability of obtaining a value of the t-statistic whose absolute value is greater than or equal to 2.64 is 0.0132.
• Therefore, the null hypothesis is rejected at the 0.05 level.

Step 1: Create Variable to Subset Population

In order to subset the data in SAS Survey Procedures, you will need to create a variable for the population of interest. In this example, the sel variable is set to 1 if the sample person is 20 years or older, and 2 if the sample person is younger than 20 years. Then this variable is used in the domain statement to specify the population of interest (those 20 years and older).

if ridageyr GE 20 then sel = 1;
else sel = 2;

Step 2: Set Up SAS Survey Procedures to Produce Means

Follow the explanations in the summary table below to produce the mean SBP using the SAS Survey procedure proc surveymeans.

Information: These programs use variable formats listed in the Tutorial Formats page. You may need to format the variables in your dataset the same way to reproduce results presented in the tutorial.

Step 3: Review SAS Means Output

• 9,056 respondents had information on systolic blood pressure (SBP).
• The results indicate the mean SBP was 124 for males and 122 for females.

Step 4: Download SAS %sregsub Macro Text File

Because version 9.1 of SAS Survey Procedures for proc surveyreg does not have a domain statement for subpopulation analyses (a domain statement is being added to proc surveyreg in SAS v9.2), you will need to use a macro provided on the SAS website. Download the file, save it to your computer, and make sure to note the location, as you will be referring your SAS program to the file later.

Link to %sregsub macro on SAS website: http://support.sas.com/ctx/samples/index.jsp?sid=483external icon

Step 5: Set up SAS Survey Procedures Macro to Test for Significance

A t-test is used to test whether the mean SBP between males and females obtained in the previous step is statistically significant different.

Request the t-test from the SAS Survey %sregsub macro and follow the steps in the summary table below.

Information: These programs use variable formats listed in the Tutorial Formats page. You may need to format the variables in your dataset the same way to reproduce results presented in the tutorial.

Information: The ( ) and the = after the macro parameters, e.g., DATA=, are part of the %sregsub macro

Step 6: Review Output

• 9,056 respondents had information on systolic blood pressure
• The number of degrees of freedom in this example is equal to 29.
• The t-statistic, with 29 degrees of freedom, is equal to 2.64. The p-value of 0.0132 indicates that the probability of obtaining a value of the t-statistic whose absolute value is greater than or equal to 2.64 is 0.0132.  Therefore, the null hypothesis is rejected at the 0.05 level.

Step 1: Create Variable to Subset Population

In order to subset the data in SAS Survey Procedures, you will need to create a variable for the population of interest. In this example, the sel variable is set to 1 if the sample person is 20 years or older, and 2 if the sample person is younger than 20 years. Then this variable is used in the domain statement to specify the population of interest (those 20 years and older).

if ridageyr GE 20 then sel = 1;
else sel = 2;

Step 2: Set Up SAS Survey Procedures to Produce Means

Follow the explanations in the summary table below to produce the mean SBP using the SAS Survey procedure proc surveymeans.

Important Note: These programs use variable formats listed in the Tutorial Formats page. You may need to format the variables in your dataset the same way to reproduce results presented in the tutorial.

Step 3: Review SAS Means Output

• 9,056 respondents had information on systolic blood pressure (SBP).
• The results indicate the mean SBP was 124 for males and 122 for females.

Step 4: Set up SAS Survey Procedures to Test for Significance

A t-test is used to test whether the mean SBP between males and females obtained in the previous step is statistically significant different.

Request the t-test from the SAS Proc Surveyreg procedure and follow the steps in the summary table below.

Important Note: These programs use variable formats listed in the Tutorial Formats page. You may need to format the variables in your dataset the same way to reproduce results presented in the tutorial.

Use SAS %sregsub Macro to Calculate Significance

Step 5: Review Output

• 9,056 respondents had information on systolic blood pressure
• The number of degrees of freedom in this example is equal to 29.
• The t-statistic, with 29 degrees of freedom, is equal to 2.64. The p-value of 0.0132 indicates that the probability of obtaining a value of the t-statistic whose absolute value is greater than or equal to 2.64 is 0.0132.  Therefore, the null hypothesis is rejected at the 0.05 level.

In this task, you will use Stata commands to calculate a t-statistic and assess whether the mean systolic blood pressures (SBP) in males and females age 20 years and older are statistically different.

Step 1: Set Up Stata to Produce Means

Follow the steps in the summary table below to produce the mean SBP and the t-test to test whether the mean SBP between males and females obtained is statistically significant different using the Stata command svy:mean.

Warning: There are several things you should be aware of while analyzing NHANES data with Stata. Please see the Stata Tips page to review them before continuing.

Step 2: Use svyset to define survey design variables

Remember that you need to define the SVYSET before using the SVY series of  commands. The general format of this command is below:

svyset [w=weightvar], psu(psuvar) strata(stratavar) vce(linearized)

To define the survey design variables for your SBP analysis, use the weight variable for 4 years of MEC data (wtmec4yr), the PSU variable (sdmvpsu), and strata variable (sdmvstra). The vce option specifies the method for calculating the variance and the default is “linearized” which is Taylor linearization.  Here is the svyset command for four years of MEC data:

svyset [w= wtmec4yr], psu(sdmvpsu) strata(sdmvstra) vce(linearized)

 Step 3:  Use svy:mean to generate means and standard errors in Stata

Now, that the svyset has been defined you can use the Stata command, svy: mean, to generate means and standard errors.  The general command for obtaining weighted means and standard errors of a subpopulation is below.

svy: mean varname, subpop(if condition)

Use the svy : mean command  with the systolic blood pressure variable (bpxsar) to estimate the mean systolic blood pressure for people age 20 years and older. Use the subpop( ) option to select a subpopulation for analysis, rather than select the study population in the Stata program while preparing the data file. This example uses an if statement to define the subpopulation based on the age variable’s (ridageyr) value. Another option is to create a dichotomous variable where the subpopulation of interest is assigned a value of 1, and everyone else is assigned a value of 0.

svy: mean bpxsar, subpop(if ridageyr>=20 & ridageyr<.)

Output of svy:mean

Step 4:  Use over option of svy:mean command to generate means and standard errors for different subgroups in Stata

You can also add the over() option to the svy:mean command to generate the means for different subgroups.  When you do this, you can type a second command, estat size, to have the output display the subgroup observation numbers.  Here is the general format of these commands for this example:

svy: mean varname, subpop(if condition) over(var1 var2)

estat size

Use the svy : mean command  with the systolic blood pressure variable (bpxsar) to estimate the mean systolic blood pressure for people age 20 years and older. Use the subpop( ) option to select a subpopulation for analysis, rather than select the study population in the Stata program while preparing the data file. This example uses an if statement to define the subpopulation based on the age variable’s (ridageyr) value. Another option is to create a dichotomous variable where the subpopulation of interest is assigned a value of 1, and everyone else is assigned a value of 0. Use the over option to get stratified results. This example produces estimates by gender. Use the estate size post estimation command to display the number of subpopulation observations and weighted numbers.

svy: mean bpxsar, subpop(if ridageyr>=20 & ridageyr<.) over(riagendr)

estat size, obs size

Output of svy:mean with over option

Step 5a: Test the hypothesis using the lincom post estimation command

If you have already done some estimations, then you can use the lincom command to test the hypothesis that the difference between the mean for the subpopulations equal 0. Use square brackets around the variable you are estimating. After the variables in square brackets, put the stratifier that you want to test (e.g. the variable in the over option). If you used labels for the variable, you can use labels instead of the coded values. Here is the general format of these commands for this example:

lincom [varname]stratval1 – [varname]stratval2

Because you have done some prior estimation, you can use the lincom  post estimation command to test the hypothesis that the difference between mean SBP (bpxsar) for males and females equal 0.  This example uses labeled values (male, female) instead of the coded values (1,2) for the gender variable (riagendr).

lincom [bpxsar]male – [bpxsar]female

Output of lincom post estimation command

Step 5b: Test the hypothesis using svy:reg command

The svy:reg command could also be used to calculate the t-statistic. The difference between using svy:reg and lincom is that svy:reg can be used without prior estimation. The xi prefix is used before the command to denote a categorical variable and the i prefix before categorical variables. Here is the general format of these commands for this example:

xi: svy, subpop(if condition): reg dependentvar i.varname

Use the svy:reg command with the xi prefix to calculate the t-statistic and assess whether the mean SBP (bpxsar) for males and females age 20 years and older are statistically different. The i prefix denotes the categorical variable, which in this example is riagendr. Use the char function choose the reference group for the categorical variable.

char riagendr[omit]2

xi:svy, subpop(if ridageyr.=20 & ridageyr<.):reg bpxsar i.riagendr,

Output of svy:reg command

Step 6: Review Stata means and t-test output

Here a table summarizing the results of the previous analyses:

According to the stratified analysis, men’s mean blood pressure is 2 points higher than women’s.  This difference is statistically significant (i.e. a difference this big or bigger would happen just by chance (in a sample of this size) only 1.3% of the time). 9,056 respondents had information on systolic blood pressure (SBP).

In this task, you will use SUDAAN to calculate a t-statistic and assess whether the mean age (ridageyr) for those who are on the 2005 Carrier File aged 65 and older is statistically different comparing participants who are obese (obese=1) and not obese (obese=0).

Step 1: Set Up SUDAAN to Produce Means

Follow the steps in the summary table below to produce the mean age using the SUDAAN procedure proc descript.

Information: These programs use variable formats listed in the Tutorial Formats page. You may need to format the variables in your dataset the same way to reproduce results presented in the tutorial.

Step 2: Review SUDAAN Means Output

• The results indicate the mean age was 74.5 for those who were not obese and on the 2005 Carrier File and 72.0 for those who were obese and on the file.

Step 3: Perform t-test to Test for Significance

A t-test is used to test whether the mean age between those who were obese and on the 2005 Carrier File and those who were not obese obtained in the previous step is statistically significant different.

Request the t-test from the SUDAAN procedure proc descript and follow the steps in the summary table below.

Information:  Note that this program and the previous program to produce means in Step 1 are identical up to the var statement.

Step 4: Review SUDAAN t-test Output

• 2,095 observations were used in the analysis size where the degrees of freedom were 44.
• To test the hypothesis that the difference between the two means is zero, the t-statistic with 44 degrees of freedom is computed as 8.82. The p-value is <0.01, which indicates that the probability of obtaining a value of the t-statistic whose absolute value is greater than or equal to 8.82 is <0.01.
• Therefore, we reject the null hypothesis at the 0.05 level.

Resources

Korn, E.L. and B.I. Graubard.  1999.  Analysis of Health Surveys. New York: Wiley.

The chi-square test is used to test the independence of two variables cross classified in a two-way table. (A chi-square statistic with n degrees of freedom is based on a statistic equal to the sum of the squares of n independent normally distributed random variables with mean=0 and unit variance.)

For example, suppose we wished to test the hypothesis that blood pressure cuff size is independent of gender and that we have the following observed frequencies obtained as a result of the cross-classification of blood pressure cuff sizes and gender.

In a simple random sample setting (unweighted data), the expected cell frequencies under the null hypothesis that blood pressure cuff size and gender are independent could be obtained by multiplying the marginal total for the jth column by the proportion of individuals in the ith row.

For example, the expected value of blood pressure cuff size 1 for men would be 285*(4312/9094)=135;  the expected value of blood pressure cuff size 4 for women would be 946*(4782/9094)=497.

Thus, if O_ij   = the observed frequency of the ith row and jth column, where i=1,2, … i and j=1,2, … j and

E_ij   = the expected frequency of the ith row and jth column

Then the formula to test the null hypothesis of independence, using the chi-square statistic, would be

Equation to Test the Null Hypothesis
 (1)

This statistic has degrees of freedom equal to the number of rows minus 1, multiplied by the number of columns minus 1.

In a complex sample setting, you would use a statistic similar to equation (1) above, modified to account for survey design with degrees of freedom equal to the number of PSUs minus the number of strata containing observations. This statistic can be obtained through SAS proc surveyfreq (CHISQ, based on the Rao-Scott chi-square with an adjusted F statistic). The analogous procedure in SUDAAN version 9.0 (proc crosstab), provides limited chi-square statistics based on Wald chi-square and does not provide an F adjusted p-value. However, SUDAAN regression models do provide F adjusted chi-square statistics which are recommended for analyzing NHANES data.

The Cochran Mantel Haenzel Test, an extension of the Pearson Chi-Square, can be applied to stratified two-way tables to test for homogeneity or independence in a non-survey setting. For a complex sample its analogue can be obtained in SUDAAN proc crosstab (cmh).

References:

Agresti A. An Introduction to Categorical Data Analysis. Wiley Series in Probability and Statistics. 1996. New York.

In this task, you will use the chi-square test to determine whether gender and blood pressure cuff size are independent of each other.

Step 1: Set Up SUDAAN to Perform Chi-Square Test

The chi-square statistic is requested from the SUDAAN procedure proc crosstab.  The summary table below provides an example of how to code for a chi-square test in SUDAAN.

Important Note: These programs use variable formats listed in the Tutorial Formats page. You may need to format the variables in your dataset the same way to reproduce results presented in the tutorial.

Important Note: SUDAAN Version 9.0 proc crosstab provides only limited chi-square results (Wald) with p-values based on unadjusted F-statistics (not the recommended statistic for complex survey data). However, the SUDAAN regression procedures do produce the recommended F adjusted chi-square statistics (e.g. Rao-Scott and Satterthwaite) for use in analyzing NHANES data.

Step 2: Review Output

• 9,094 respondents have information on blood pressure cuff size.
• The row percentages indicate that males tend to have a larger cuff size than females.
• Because the p-value is less than 0.05, you would reject the null hypothesis that gender and blood pressure cuff size are independent. The probability of obtaining a value of 274.74 or more is approximately zero.

In this task, you will use the chi-square test in SAS to determine whether gender and blood pressure cuff size are independent of each other.

Step 1: Set Up SAS to Perform Chi-Square Test

The chi-square statistic is requested from the SAS Survey Procedures procedure proc surveyfreq.  The summary table below provides an example of how to code for a chi-square test in SAS.

Important Note: These programs use variable formats listed in the Tutorial Formats page. You may need to format the variables in your dataset the same way to reproduce results presented in the tutorial.

Important Note: For complex survey data such as NHANES, we recommend using the Rao-Scott F adjusted chi-square statistic since it yields a more conservative interpretation than the Wald chi-square.

Step 2: Review output

• 9,094 respondents have information on blood pressure cuff size.
• The row percentages indicate that males tend to have a larger cuff size than females.
• Because the F adjusted p-value is less than 0.05, you would reject the null hypothesis that gender and blood pressure cuff size are independent. The probability of obtaining a value of 125.55 or more is approximately zero.

In this task, you will use the chi-square test in Stata to determine whether gender and blood pressure cuff size are independent of each other. The chi-square statistics is requested from the Stata command svy:tabulate.

Warning: There are several things you should be aware of while analyzing NHANES data with Stata. Please see the Stata Tips page to review them before continuing.

Step 1: Use svyset to define survey design variables

Remember that you need to define the SVYSET before using the SVY series of  commands. The general format of this command is below:

svyset [w=weightvar], psu(psuvar) strata(stratavar) vce(linearized)

To define the survey design variables for your blood pressure cuff size (bpacsz) analysis, use the weight variable for four-yours of MEC data (wtmec4yr), the PSU variable (sdmvpsu), and strata variable (sdmvstra) .The vce option specifies the method for calculating the variance and the default is “linearized” which is Taylor linearization.  Here is the svyset command for four years of MEC data:

svyset [w= wtmec4yr], psu(sdmvpsu) strata(sdmvstra) vce(linearized)

Step 2: Regroup blood pressure cuff size variable

In this example, a new variable (cuff_size) is created to regroup blood pressure cuff size (bpacsz) from five categories to four categories. This collapses the infant (1) and child (2) groups. Use the gen command to create a new variable.

gen cuff_size=1 if bpacsz==1 | bpacsz==2
replace cuff_size=2 if bpacsz==3
replace cuff_size=3 if bpacsz==4
replace cuff_size=4 if bpacsz==5

Step 3: Generate chi-square statistics using svy:tabulate

Now, that the svyset has been defined you can use the Stata command, svy: tabulate, to produce two-way tabulations with tests of independence.  Some of the options for the tab command include:

• column and row to display column and row percentages (if you do not specify this you will get cell proportions);
• obs lists the number of observations in each cell; count lists the weighted n in each cell and by adding format(%11.0fc) you will display the counts with commas rather than scientific notation;
• ci gives the confidence interval around each estimate, but can only be used with either row or column, not both; and
• the Pearson (Rao-Scott correction F-statistic) chi-square (pearson), null-based (null), and Wald (wald) test statistics.

The general command for generating two-way tabulations is below.

svy:tabulate varname, subpop(if condition) options

Use the svy : tabulate command  to produce two-way tabulations for gender (riagendr) and blood pressure cuff size (cuff_size) with tests of independence for people age 20 years and older. (See Section 5.4 of Korn and Graubard Analysis of Data from Health Surveys, pp 207-211).  Use the subpop( ) option to select a subpopulation for analysis, rather than select the study population in the Stata program while preparing the data file. This example uses an if statement to define the subpopulation based on the age variable’s (ridageyr) value. Another option is to create a dichotomous variable where the subpopulation of interest is assigned a value of 1, and everyone else is assigned a value of 0. The options specified for this example, use the column, rows, obs, percent, pearson, null and wald test statistic options.

svy:tab riagendr cuff_size, subpop (if ridageyr >=20 & ridageyr<.) column row obs percent pearson null wald

Output of svy:tabulate command with column, row, obs, percent, pearson, null and wald options

Step 4: Review output

Here is a table summarizing the output:

Men have a larger cuff size than women – for example, 70% of men had cuff size of 4 or 5 compared to 51% of women.  Cuff size varies significantly according to gender (p<0.0001).  NOTE:  The grayed cells have too few observations to create stable estimates and should probably not be reported.

In this task, you will use the chi-square test to determine whether obesity is associated with gender among those who were age 65 and older at the NHANES examination and have claims on the 2005 Carrier File.

Step 1: Set Up SUDAAN to Perform Chi-Square Test

The Chi-square statistic is requested from the SUDAAN procedure proc crosstab.  The summary table below provides an example of how to code for a chi-square test in SUDAAN.

Information: These programs use variable formats listed in the Tutorial Formats page. You may need to format the variables in your dataset the same way to reproduce results presented in the tutorial.

Information: SUDAAN Version 9.0 proc crosstab provides only limited Chi-square results (Wald) with p-values based on unadjusted F-statistics (not the recommended statistic for complex survey data). However, the SUDAAN regression procedures do produce the recommended F adjusted chi-square statistics (e.g. Rao-Scott and Satterthwaite) for use in analyzing NHANES data.

Step 2: Review Output

• 2,095 respondents were included in this analysis.
• The row percentages do not indicate a difference in obesity status by gender for those on the 2005 Carrier File.
• Because the p-value is greater than 0.05, we fail to reject the null hypothesis at the 0.05 level and it is concluded that there is no statistically significant association of obesity and gender for those on the 2005 Carrier File.

Resources

Korn, E.L. and B.I. Graubard.  1999.  Analysis of Health Surveys. New York: Wiley.