SAS II

Inferential Statistics

After attending this training session, you will be able to run and interpret the following tests in SAS:

Chi-square test
Independent samples t-test and Levene’s test for the equality of variances
One-way ANOVA
Bivariate Statistics: Correlation
Bivariate Statistics: Regression
Multiple regression

Introduction

Code and Dataset

For this training session, please download the following data and code:

gss2018sasi.sas7bdat

SAS II Script for Training.sas

Recap of SAS I

In the SAS I training session, we covered the following topics. Please feel free to refer back to the SAS I workbook using the following links if you would like a refresher.

As in the SAS I training session, we will be using data from the General Social Survey.

The Data for This Workshop

The SAS program below creates a temporary dataset using data from the 2018 GSS, gss2018sasi.sas7bdat. Create a folder called Training in which you will save the SAS gss2018sasi data file. Next, we will set up a working directory called Training for our analysis. Go to the Explorer and navigate to Libraries. Right click and create a new library called Training using the location for your training folder. For step by step instructions, see the SAS I Workbook. We will now create a temporary dataset for this training, gss2018.

data gss2018;
    set training.gss2018sasi;
run;

Chi-square Test

A chi-square test is used to determine whether there is a statistically significant association between two categorical variables. The chi-square test statistic is requested in the proc freq procedure.

Example 1: 2x2 Table

For example, you may want to know if there is a significant association between the variables sex (Respondent’s gender) and hschem (Respondent ever took chemistry class in high school). The following program runs proc freq with the chisq option:

proc freq data = gss2018;
      tables hschem*sex /chisq;
run;

The variable name that appears immediately after tables will appear in the rows and the second variable, which appears after the *, will be placed in the columns. Either of the variables can go in the rows or columns.

SAS generates the following test statistics for chi-square. The levels of measurement, number of categories within the variables, and the sample size will determine which test statistic is appropriate.

Chi-Square: Used to test the hypothesis that the row and column variables are independent. This should not be used if any cell has an expected value less than 1, or if more than 20 percent of the cells have expected values less than 5.

Likelihood Ratio Chi-Square: A goodness-of-fit statistic similar to Pearson’s chi-square. This statistic is appropriate to report for smaller sample sizes. The likelihood ratio approaches the Pearson’s chi-square as n increases, so for large sample sizes, the two statistics are the same and either can be reported.

Continuity Adj. Chi-Square: A correction is applied in the computation of chi-square statistic for 2x2 tables to improve the approximation. Corrected chi-square values are always smaller than uncorrected values.

Mantel-Haenszel Chi-Square: A measure of the linear association between the row and column variables. This statistic should not be used for nominal data. It is also referred to as the Linear-by-Linear Association and the Mantel-Haenszel test for linear association.

The Phi Coefficient, Contingency Coefficient, and Cramer’s V are measures of the strength of the association and are not covered in this workshop.

Fisher’s Exact Test: A test for independence in a 2x2 table. This statistic is most useful when the total sample size and expected values are small.

Because this example uses a 2X2 table, we will use Fisher’s Exact test. Here, the Fisher’s Exact test has a probability p-value larger than 0.05 so we can reject the null hypothesis using the two-sided (or two-tailed) p-value. Thus, we can conclude that there is a statistically significant relationship between the respondent’s gender and taking chemistry in high school.

Example 2: 3x2 Table

In this example, you test whether there is a significant association between the variables natspac (attitude towards the space exploration program) and colsci (whether respondent took college-level science courses).

The following program runs proc freq with the chisq option:

proc freq data = gss2018;
      tables natspac*colsci /chisq;
run;

It is appropriate to use the Chi-Square test statistic in this example. The p-value is smaller than 0.0001. Therefore, it may be concluded that there is a statistically significant relationship between attitude towards the space exploration program and having taken college level science classes.

Additional Options for Contingency Tables

Some additional options that can be specified on the tables statement (appearing after the forward slash - /) in proc freq include:

all - calculates all available statistics in a two-way table
chisq - calculates chi-square and other statistics for independence in a two-way table
expected - computes the expected counts for a two-way table
fisher - requests Fisher’s exact test for tables larger than two-by-two
nocol - omits column percents
norow - omits row percents
nopercent - omits table total percents

Exercise 1

Perform a chi-square test to determine if educational attainment (degree) is significantly associated with the belief that marijuana should be legal (grass). (See Appendix A for answers.)

How many high school graduates (degree = 1) said Yes to legalization (grass = 1)?
Is there a statistically significant relationship between the two variables? Which chi-square test statistic is appropriate?

Independent samples t-test

An independent samples t-test determines whether there is a statistically significant difference in the mean of a continuous variable between two groups. The independent samples t-test is obtained by using the proc ttest procedure.

For example, this procedure can be used to find out if there is a statistically significant difference between the responses of men and women (sex) for r how much they have to relax per day (hrlax). The following program runs proc ttest:

proc ttest data = gss2018;
      class sex;
      var hrlax;
run;

The grouping variable (sex) is identified in the class statement. The test variable (hrlax) is identified in the var statement.

The results of the t-test presented in the output are used to decide whether there is a statistically significant difference by sex in hours relaxed per week. There are two sets of statistics generated by SAS: (1) one set that assumes the variances in the distribution of the dependent variable are equal for both groups - Pooled method; and (2) the other set that does not assume that the variances in the distributions are equal - Satterthwaite method.

To determine which method is appropriate, check the p-value of the Equality of Variances test results. When the Equality of Variances test generates a non-significant F-test statistic, the assumption of equal variance is met, and the statistics for Pooled method should be used. Because the p-value of Equality of Variances test in this example is larger than 0.05 (p=0.2834), the Pooled statistics should be used to determine if there is a statistically significant difference.

Using the Pooled test, the p-value of the t statistic is less than 0.05, therefore we may conclude that there is a statistically significant difference in the number of hours men and women have to relax per day.

Exercise 2

Find out if there is a statistically significant difference in terms of respondent’s family income(fmi) between respondents who took college-level science courses and those who didn’t? (colsci). (See Appendix A for answers.)

ANOVA

A One-way ANOVA is used to determine whether there is a statistically significant difference in the mean of a continuous dependent variable among (typically more than) two groups. One method of performing ANOVA in SAS is to use the proc glm procedure. Unlike proc anova, the proc glm procedure is designed to handle both balanced data (data with equal numbers of observations for every combination of the classification factors) and unbalanced data.

For example, this procedure may be used to find out if there is a statistically significant difference in the average number of hours spent watching television per day (tvhours) by marital group (marital). The following program runs proc glm with the hovtest option:

proc glm data = gss2018;
      class marital;
      model tvhours = marital;
      means marital / hovtest;
run;

The dependent variable (tvhours) and grouping variable (marital) are both identified in the model statement in the expression tvhours = marital. The grouping variable (marital) is also identified in both the class and means statements. The means statement is used to obtain descriptive statistics for the dependent variable by the grouping variable. The hovtest option is specified in the means statement to request the Levene’s test of the homogeneity of variance. Before we check the ANOVA results, we should check Levene’s test results.

From the Levene results table, P value is smaller than 0.05. We reject the null hypothesis that the variances are equal, and not-equal variances are assumed. The result confirms that We should use Welch test for non-equal variances, instead of regular ANONVA results shown from this outcome.

Welch Test

When the homogeneity of variances assumption is not met, the Welch test should be requested when performing one-way ANOVA. To request the Welch test of equality of means, specify the welch option in the means statement.

The following program runs proc glm with the welch option:

proc glm data = gss2018;
      class marital;
      model tvhours = marital;
      means marital / welch;
run;

From the Welch table, you can conclude that the difference in the mean number of hours watching television per day among marital groups is statistically significant.

ANOVA Post Hoc Tests

Once you have determined that differences exist among the means, post hoc tests can determine which means differ. Post hoc tests are specified in the means statement. Post hoc tests available in SAS include:

Keyword	Test
`t`	Student’s t
`snk`	Student-Newman-Keuls
`tukey`	Tukey-Kramer
`sidak`	Sidak
`gabriel`	Gabriel
`duncan`	Waller-Duncan
`regwq`	Ryan-Einot-Gabriel-Welch
`bon`	Bonferroni
`scheffe`	Scheffe
`dunnett`	Dunnett

Note

In SAS only the Dunnett test is available for pairwise comparisons for one-way ANOVA when variances in groups are not equal.

In this example, looking at differences in the average time spent watching TV per day (tvhours) by marital status (marital), the results of the Levene’s Test of Homogeneity of TVHOURS Variance indicate that equal variances can not be assumed. Therefore, we can select a post hoc test that assumes equal variances. For example, specify dunnett in means statement to request the Dunnett post hoc test.

The following program runs proc glm with the dunnett option:

proc glm data = gss2018;
      class marital;
      model tvhours = marital;
      means marital / dunnett;
run;

Note

In the data, Married = 1, Widowed = 2, Divorced = 3, Separated = 4, and Never married = 5.

The Dunnet Tests for TVHOURS provides the results of post hoc test. From this table it may be concluded that:

Group 1 (Married) is significantly different from Group 2 (Widowed) and Group 3 (Divorced) at the 0.05 level.
All the other groups based on marital status have no significant different with each other. After identifying statistically significant differences, We can determine which group watched more TV hours per day: Compared with Married group, Widowed group watched 1.38 more hours and Divorced group watched 0.579 more hours per day.

Exercise 3

Using the one-way ANOVA procedure, test whether respondents’ family income(fmi) is different among different education groups (degree). Remember to check the Test of Homogeneity of Variances to determine if the Welch test is required. (See Appendix A for answers.)

Bivariate Correlation

Correlation is a measure of the linear relationship between two continuous variables. A correlation coefficient ranges from -1 to 1. A positive correlation coefficient indicates a positive linear relationship (i.e., as one variable increases, the other tends to as well). On the other hand, a negative correlation coefficient indicates a negative linear relationship. A correlation coefficient of 0 indicates there is no linear relationship between the two variables. Bivariate correlations are obtained using the proc corr procedure.

For example, this procedure may be used to measure the relationship between the continuous variables spouse occupation prestige score (sop), spouse socioeconomic index score (ssi), and no.of hrs spouse usually works a week (sphrs2).

Note

A bivariate correlation measures the linear relationship between two continuous variables, but it is convenient to calculate several bivariate correlations simultaneously in a matrix.

The following program runs proc corr:

proc corr data = gss2018;
      var ssi sop sphrs2;
run;

Each cell contains:

A correlation coefficient (Pearson Correlation)
The p-value associated with the correlation coefficient for the test of “no linear relationship”
The number of cases (N)

In this example, the correlation coefficient between sop and ssi is 0.83556 and the significance of the coefficient is less than $0.0001$, which suggests that these two variables are significantly and positively correlated.

Note on missing values and sample size (N): According to the third row of the cell for the correlation between sphrs2 with ssi or sop , only 22 cases are used, while there are 1,102 valid cases for both ssi or sop variables. The default treatment of missing cases in the Correlation procedure is to use the maximum number of non-missing values for each pair of variables, which is known as pairwise deletion. An alternative treatment of missing cases is listwise deletion. When this option is selected, cases with a missing value in any of the analyzed variables will be excluded from the analysis. (When only two variables are included, the sample size is the same whether pairwise or listwise deletion is used. However, these options make a difference in the sample size when three or more variables are included.)

The default is pairwise deletion, but we can change this by adding the nomiss option to the proc corr procedure. The interpretation of the correlation coefficient for listwise deletion is the same as that for pairwise deletion. The following program runs proc corr with the nomiss option:

proc corr data = gss2018 nomiss;
      var ssi sop sphrs2;
run;;
run;

Exercise 4

Compute the correlation between: the highest year of school completed (educ), the respondent’s father’s occupational prestige index (faos), and the respondent’s mother’s occupational prestige index (moos) using (1) pairwise deletion and (2) listwise deletion and compare how the sample size changes. (See Appendix A for answers.)

Bivariate Regression

Bivariate linear regression is used to assess the influence of a continuous or dichotomous independent variable on a continuous dependent variable. Regression analysis is performed using the proc reg procedure.

This example uses bivariate regression to measure the effect of educational attainment (educ) on family’s income (realinc).

proc reg data = gss2018;
      model realrinc = educ /stb;
run;

The option stb requests standardized coefficients.

Interpretation of Results:

Significance: The Analysis of Variance table tells is if the independent variable has a significant linear relationship with the dependent variable. In this example, the F-statistic is statistically significant (p-value <0.001). Thus, educational attainment is a significant predictor of income.
Model Fit: R-squared, also known as the coefficient of determination, tells you the proportion of variation in the dependent variable explained by the independent variable. R-squared ranges from 0 (no relationship) to 1 (perfect prediction of dependent variable from independent variable). In this example, 9.24% of the variation in income (dependent variable) is explained by the years of education (independent variable).

An Adjusted R-squared is adjusted for the number of independent variables used in the model. However, since there is only one independent variable in this example, R-squared and Adjusted R-squared should be similar.
Parameter Estimates: This table provides the values of the unstandardized regression coefficients (Parameter Estimate), standardized coefficients (Standardized Estimate), and a significance test of the coefficients (Pr > |t|).

The regression coefficient for the intercept is -19,272, while the coefficient for the independent variable educ is 3,131.35065. It can be said that the predicted level of income will increase by $3,131.35 for each additional year of education. Because this effect is measured with respect to the original scale of the independent variable, it is referred to as an unstandardized coefficient. When we have multiple independent variables as predictors in the model, we cannot compare the effects of independent variables using unstandardized coefficients. This will be addressed further in the section on Multiple Regression.

The standard error (SE) is used to calculate the t-statistic ($\beta/SE$). This statistic is used to test the null hypothesis that the coefficient is equal to 0. In this example, educ is statistically significant at the 0.0001 level. Thus, you can reject the null hypothesis that the education variable’s coefficient is equal to 0 and conclude education is a significant predictor of income.

Exercise 5

Does age (age) have a significant linear relationship with performance on a vocabulary test (wordsum)? Treat wordsum as the dependent variable. (See Appendix A for answers.)

Multiple Regression

Multiple regression is the same as bivariate regression, except it uses two or more independent variables to explain the variation of the dependent variable. This example uses Multiple regression to measure the effect of respondents’ health score (rhs) and age (age) on respondent’s income (realrinc).

This example uses multiple regression to measure the effect of educational attainment (educ), sex(male), and age (age) on family’s income (realinc).

proc reg data = GSS2018;
      model realrinc = rhs age /stb;
run;

The dependent variable (realrinc) is placed in the model statement before the equal sign.
The independent variables (age, and rhs) are placed in the model statement after the equal sign.
The stb option is specified on the model statement to obtain standardized regression coefficients in addition to unstandardized regression coefficients.

Significance: The Analysis of Variance table measures whether the linear relationship between the independent variables and dependent variable is significant. In this example, the F-statistic is below 0.05. Thus, the groups of three variables significantly predicts income.
Model Fit: Since the denominator of R-squared is fixed, only the numerator can increase. Thus, each additional variable used in the equation increases the size of the numerator only. As a result, the introduction of additional variables produces a higher R-squared value, regardless of their usefulness. The Adjusted R-Squared value corrects this problem by adjusting both the numerator and the denominator by their respective degrees of freedom. Unlike R-squared, the Adjusted R-squared can decrease as more predictors are added to a model. It is useful when comparing models, because it balances the goals of model fit and model simplicity.

In this example, compared to the bivariate model in the previous section, both R-squared and Adjusted R-squared have increased. Hence, this new model explains more variation of the dependent variable than the old model with one independent variable, and the additional variation explained is worth the cost of making the model more complex.
Parameter Estimates: The Parameter Estimates tables provide the values of the unstandardized regression coefficients (Parameter Estimate), standardized coefficients (Standardized Estimate), and a significance test of the coefficients (Pr > |t|). It shows that a unit increase in rhs will decrease predicted income by $3809.57 (higher rhs means worse health condition) while holding age constant. The t-test statistic is -2.59, and the p-value associated with the t-test statistic is`<0.01. Thus, rhs is statistically significant at the 0.01 level while controlling for age.

To compare the magnitude of the effect of each independent variable, the coefficients must be standardized using the same scale. The standardized coefficient (Standardized Estimate) measures change in the dependent variable (measured in standard deviations) per one standard deviation change in the independent variable.

Because the absolute value of standardized coefficient for age is larger than that for rhs we can conclude that age has a larger impact on real income than health ratings.

Exercise 6

Run multiple regression to assess the effect of occupational prestige index of a respondent’s parents (faos and moos) and respondent’s gender (Male) on the respondent’s occupational prestige (pres). (See Appendix A for answers.)

Plotting a Regression Line on a Scatter Plot

The relationship between respondents’ education and income can be visually portrayed in a scatter plot, using the proc gplot procedure. In addition, you can overlay the regression line and confidence interval range on the scatter plot.

symbol color = black value = star interpol = r;
title "Age against R's Income";
proc gplot data = gss2018;
      plot  realrinc*age;
run;
quit;

symbol statements give proc gplot information about plot characters, plot lines, color, and interpolation (smoothing of lines).
The color black is specified in the color option.
The character star is specified in the value option
The interpol=r option indicates that the plot is a regression analysis.
The dependent variable (realrinc) appears first in the plot statement, before the *, and the independent variable (age) appears second, after the *.

Important

If a second symbol is present in a plot, the symbol statement must include a reference to which symbol is being defined by adding a number to symbol. The first symbol would be referred to as symbol1 and the second symbol would be referred to as symbol2.

APPENDIX A: ANSWERS FOR EXERCISES

SAS II Complete Script

Exercise 1

Perform a chi-square test to determine if educational attainment (degree) is significantly associated with the belief that marijuana should be legal (grass). * How many high school graduate (degree = 1) said Yes to legalization (grass = 1)? * Is there a statistically significant relationship between the two variables? Which chi-square test statistic is appropriate?

proc freq data = gss2018;
      tables grass*degree / chisq expected;
run;

438 high school graduates said “Yes” to legalization of marijuana.
There is a statistically significant relationship between the two variables. The p-value associated with the chi-square statistic (0.0141) is less than 0.05. We know that the chi-square test is appropriate because the expected cell count for all cells exceeds five.

Exercise 2

Find out if there is a statistically significant difference in terms of respondents family income (fmi) between resondents who took college-level science courses and those who did not (colsci).

proc ttest data = GSS2018;
      class colsci;
      var fmi;
run;

Because the Equality of Variances test is significant $(0.0001 < 0.05)$, the Satterthwaite test is appropriate.
The Satterthwaite t-test statistic is significant $(0.0001 < 0.05)$, therefore the amount of income respondents made differs significantly based on if they did or did not take college-level science courses.

Exercise 3

Using the one-way ANOVA procedure, test whether respondents family income (fmi) is different among different education groups (degree). Remember to check the Test of homogeneity of variance to determine if the Welch test is required.

proc glm data = gss2018;
      class degree;
      model fmi = degree;
      means degree / hovtest;
run;

The dependent variable (fmi) and grouping variable (degree) are both identified in the model statement in the expression fmi=degree.
The grouping variable (degree) is identified in both the class and means statements. The means statement is used to obtain descriptive statistics for the dependent variable by grouping variable.
The hovtest option is specified in the means statement to request the Levene’s test of the homogeneity of variance.

From the Levene’s results table, p-value is smaller than 0.05. We reject the null hypothesis that the variances are equal, and not-equal variances are assumed. The result confirms that we should use the Welch test for non-equal variances, instead of regular ANOVA procedure results.

proc glm data = gss2018;
      class degree;
      model fmi = degree;
      means degree / welch;
run;

From the results above,we can conclude that differences in family income among educational levels are statistically significant.

Exercise 4

Pairwise Deletion:

proc corr data = gss2018;
      var educ faos moos;
run;

Educational attainment (educ) is significantly and positively correlated with mother’s and father’s occupational prestige index. Mother’s and father’s occupational index are also significantly and positively correlated.

Listwise Deletion:

proc corr data = GSS2018 nomiss;
      var educ faos moos;
run;

While the directions of the relationships between the variables do not change significantly if you use either pairwise or listwise deletion, the values of the correlation coefficients do change depending on the treatment of missing cases.

Pairwise deletion should include more samples, because it includes more cases based on how it treats missing variables.

Exercise 5

Does age (age) have a significant linear relationship with performance on a vocabulary test (wordsum)? Treat wordsum as the dependent variable.

proc reg data = gss2018;
      model wordsum = age /stb;
run;

Because the F-statistic is statistically significant, age is a significant predictor of vocabulary test score at the 0.0001 level.
.6 percent of the variation in vocabulary test score is explained by age.
The regression coefficient for independent variable age is 0.00921 and the constant for the model is 5.4726 The predicted vocabulary test score will increase by 0.00921 for each additional year of age.

Exercise 6

Run multiple regression to assess the effect of the occupational prestige index of a respondent’s parents (faos and moos) and respondent’s gender (Male) on the respondent’s occupational prestige (pres).

proc reg data = gss2018;
      model pres = moos faos male /stb;
run;

Because the F-statistic is statistically significant, as a group, mother’s occupational prestige index score, father’s occupational prestige index score, and sex are significant predictors of respondent’s occupational prestige score.
The predicted occupational prestige score will increase by 0.166 for each unit increase of mother’s occupational prestige score, 0.142 for each unit increase of father’s occupational prestige score, and -0.224 if the respondent is male.
The effects of faos on moos are statistically significant at the 0.001 level. The effect of sex (male) is not statistically significant.
5.58 percent of the variation in pres is explained by the three predictors.