Masters Level Quantitative Design Assessment T-Scores

Assessment 3 Context

You will review the theory, logic, and application of t-tests. The t-test is a basic inferential statistic often reported in psychological research. You will discover that t-tests, as well as analysis of variance (ANOVA), compare group means on some quantitative outcome variable.

Recall that null hypothesis tests are of two types: (1) differences between group means and (2) association between variables. In both cases there is a null hypothesis and an alternative hypothesis. In the group means test, the null hypothesis is that the two groups have equal means, and the alternative hypothesis is that the two groups do not have equal means. In the association between variables type of test, the null hypothesis is that the correlation coefficient between the two variables is zero, and the alternative hypothesis is that the correlation coefficient is not zero.

Notice in each case that the hypotheses are mutually exclusive. If the null is false, the alternative must be true. The purpose of null hypothesis statistical tests is generally to show that the null has a low probability of being true (the p value is less than .05) – low enough that the researcher can legitimately claim it is false. The reason this is done is to support the allegation that the alternative hypothesis is true.

In this context you will be studying the details of the first type of test. This is the test of difference between group means. In variations on this model, the two groups can actually be the same people under different conditions, or one of the groups may be assigned a fixed theoretical value. The main idea is that two mean values are being compared. The two groups each have an average score or mean on some variable. The null hypothesis is that the difference between the means is zero. The alternative hypothesis is that the difference between the means is not zero. Notice that if the null is false, the alternative must be true. It is first instructive to consider some of the details of groups. Means, and difference between them.

Null Hypothesis Significance Test

The most common forms of the Null Hypothesis Significance Test (NHST) are three types of t tests, and the test of significance of a correlation. The NHST also extends to more complex tests, such as ANOVA, which will be discussed separately. Below, the null hypothesis and the alternative hypothesis are given for each of the following tests. It would be a valuable use of your time to commit the information below to memory. Once this is done, then when we refer to the tests later, you will have some structure to make sense of the more detailed explanations.

1. One-sample t test: The question in this test is whether a single sample group mean is significantly different from some stated or fixed theoretical value – the fixed value is called a parameter.

· Null Hypothesis: The difference between the sample group mean and the fixed value is zero in the population.

· Alternative hypothesis: The difference between the sample group mean and the fixed value is NOT zero in the population.

2. Dependent samples t test (also known as correlated groups t or repeated measures t): The question in this test is whether two scores for each participant differ significantly. It is actually a special case of the one-sample test, where each person’s score is the difference between his or her two original scores (difference scores). If there is no significant difference in the population, then the mean population difference score is zero (the fixed value).

· Null Hypothesis: The mean difference between the two scores for each participant is zero in the population.

· Alternative hypothesis: The mean difference between the two scores for each participant is NOT zero in the population

3. Independent samples t test (two independent groups): The question in this test is whether or not two group means are from the same population, or from populations with different means.

· Null Hypothesis: The difference between the two group’s means is zero in the population, or the two groups are from the same population.

· Alternative hypothesis: The difference between the two group’s means is NOT zero in the population, or the two groups are from different populations.

Logic of the t-Test

Imagine that a school psychologist compares the mean IQ scores of Class A versus Class B. The mean IQ for Class A is 102.0 and the mean IQ for Class B is 105.0. Is there a significant difference in mean IQ between Class A and Class B?

To answer this question, the school psychologist conducts an independent samples t-test. The independent samples t-test compares two group means in a between-subjects (between-S) design. In this between-S design, participants in two independent groups are measured only once on some outcome variable. By contrast, a paired samples t-test compares group means in a within-subjects (within-S) design for one group. Each participant is measured twice on some outcome variable, such as a pretest-posttest design. For example, a school psychologist could measure self-esteem for a class of students prior to taking a public speaking course (pretest) and then measure self-esteem again after completing the public speaking course (posttest). The paired samples t-test determines if there is a significant difference in mean scores from the pretest to the posttest.

Focus on the logic and application of the independent samples t-test. There are two variables in an independent samples t-test: the predictor variable (X) and the outcome variable (Y). The predictor variable must be dichotomous, meaning that it can only have two values (for example, male = 1; female = 2). Notice this is nominal level variable. The outcome variable must be at the interval level or above (ratio). Group membership is mutually exclusive. In nonexperimental designs, group membership is based on some naturally occurring characteristic of a group (for example, gender). In experimental designs, participants are randomly assigned to one of two group conditions (for example, treatment group = 1; control group = 2). In contrast to the dichotomous (nominal) predictor variable, the outcome variable must be quantitative to calculate a group mean (for example, mean IQ score, mean heart rate score).

Assumptions of the t-Test

All inferential statistics, including the independent samples t-test, operate under assumptions checked prior to calculating the t-test in SPSS. Violations of assumptions can lead to erroneous inferences regarding a null hypothesis. The first assumption is independence of observations. For predictor variable X in an independent samples t-test, participants are assigned to one and only one “condition” or “level,” such as a treatment group or control group. This assumption is not statistical in nature; it is controlled by proper research procedures that maintain independence of observations.

The second assumption is that outcome variable Y is quantitative and normally distributed. This assumption is checked by a visual inspection of the Y histogram and calculation of skewness and kurtosis values. A researcher may also conduct a Shapiro-Wilk test in SPSS to check whether a distribution is significantly different from normal. The null hypothesis of the ShapiroWilk test is that the distribution is normal. If the Shapiro-Wilk test is significant, then the normality assumption is violated. In other words, a researcher wants the Shapiro-Wilk test to not be significant at p < .05.

The third assumption is referred to as the homogeneity of variance assumption. Ideally, the amount of variance in Y scores is approximately equal for group 1 and group 2. This assumption is checked in SPSS with the Levene test. The null hypothesis of the Levene test is that group variances are equal. If the Levene test is significant, then the homogeneity assumption is violated. In other words, a researcher wants the Levene test to not be significant at p < .05.

SPSS output for the t-test provides two versions of the t-test: “Equal variances assumed” and “Equal variances not assumed.” If the Levene test is not significant, researchers report the “Equal variances assumed” version of the t-test. If the Levene test is significant, researchers report the more conservative “Equal variances not assumed” calculation of the t-test in the second row of the output table. Masters Level Quantitative Design Assessment T-Scores

Hypothesis Testing for a t-Test

The null hypothesis for a t-test predicts no significant difference in population means, or H0: µ1 = µ2. A directional alternative hypothesis for a t-test is that the population means differ in a specific direction, such as H1: µ1 > µ2 or H1: µ1 < µ2. A non-directional alternative hypothesis simply predicts that the population means differ, but it does not stipulate which population mean is significantly greater (H1: µ1 ≠ µ2). For t-tests, the standard alpha level for rejecting the null hypothesis is set to .05. SPSS output for a t-test showing a p value of less than indicates that the null hypothesis should be rejected; there is a significant difference in population means. A p value greater than .05 indicates that the null hypothesis should not be rejected; there is not a significant difference in population means.

Effect Size for a t-Test

There are two commonly reported estimates of effect size for the independent samples t-test, including eta squared (η2) and Cohen’s d . Eta squared is analogous to r2. It estimates the amount of variance in Y that is attributable to group differences in X. Eta squared ranges from 0 to 1.0, and it is interpreted similarly to r2 in terms of “small,” “medium,” and “large” effect sizes. Eta squared is calculated as a function of an obtained t value and the study degrees of freedom.

Cohen’s d is an alternate effect size representing the number of standard deviations the two population means are in the sample. A small Cohen’s d (< .20) indicates a high degree of overlap in population means. A large Cohen’s d (> .80) indicates a low degree of overlap in population means.

Testing Assumptions: The Shapiro-Wilk Test and the Levene Test

Recall that two assumptions of the t-test are that:

4. Outcome variable Y is normally distributed.

5. The variance of Y scores is approximately equal across groups (homogeneity assumption).

The Shapiro-Wilk Test

In addition to a visual inspection of histograms and skewness and kurtosis values, SPSS provides a formal statistical test of normality referred to as the Shapiro-Wilk test. A perfect normal distribution will have a Shapiro-Wilk value of 1.0. Values less than 1.0 indicate an increasing departure from a perfect normal shape. The null hypothesis of the Shapiro-Wilk test is that the distribution is normal. When the Shapiro-Wilk test indicates a p value less than .05, the normality assumption is violated. Masters Level Quantitative Design Assessment T-Scores

To obtain the Shapiro-Wilk test, in SPSS select “Analyze…Descriptive Statistics…Explore.” Place the outcome variable Y in the “Dependent List” box and select the “Plots” option. Select the “Normality plots with tests” option. Press “Continue” and then “Ok.” SPSS provides the Shapiro-Wilk test output for interpretation. A significant Shapiro-Wilk test ( p < .05) suggests that the distribution is not normal and interpretations may be affected. However, the t-test is fairly robust to violations of this assumption when sample sizes are sufficiently large (that is, > 100).

The Levene Test

The homogeneity of variance assumption is tested with Levene test. The Levene test is automatically generated in SPSS when an independent samples t-test is conducted. The null hypothesis for the Levene test is that group variances are equal. A significant Levene test ( p < .05) indicates that the homogeneity of variance assumption is violated. In this case, report the “Equal variances not assumed” row of the t-test output. This version of the t-test uses a more conservative adjusted degrees of freedom ( df) that compensates for the homogeneity violation. The adjusted df can often result in a decimal number (for example, df = 13.4), which is commonly rounded to a whole number in reporting (for example, df = 13). If the Levene test is not significant (that is, homogeneity is assumed), report the “Equal variances assumed” row of the t-test output. Masters Level Quantitative Design Assessment T-Scores

Proper Reporting of the Independent Samples t-Test

Reporting a t-test in proper APA style requires an understanding of the following elements, including the statistical notation for an independent samples t-test (t), the degrees of freedom, the t value, the probability value, and the effect size. To provide context, provide the means and standard deviations for each group. For example, imagine an industrial/organizational psychologist randomly assigns 9 employees to a treatment group (for example, team-bonding exercises) and 9 employees to a control group (for example, no exercises) and then subsequently measures their rates of organizational citizenship behavior (OCB) over a period of six months. The results show:

The mean OCB scores differed significantly across groups, t(16) = -2.58, p = .02 (two-tailed). Mean OCB for the control group (M = 67.8, SD = 8.2) was about 10 OCB points lower than mean OCB for the treatment group (M = 77.9, SD = 8.1). The effect size, as indexed by η2 was .30; this is a very large effect.

t, Degrees of Freedom, and t Value

The statistical notation for an independent samples t-test is t, and following it is the degrees of freedom for this statistical test. The degrees of freedom for t is n1 + n2 – 2, where n1 equals the number of participants in group 1 and n2 equals the number of participants in group 2. In the example above, N = 18 (n1 = 9; n2 = 9). The t value is a ratio of the difference in group means divided by the standard error of the difference in sample means. The t value can be either positive or negative.

Probability Value

A researcher estimates the probability value based on a table of critical values of t for rejecting the null hypothesis. In the example above, with 16 degrees of freedom and alpha level set to .05 (two-tailed), the table indicates a critical value of +/- 2.12 to reject the null hypothesis. The obtained t value above is -2.58, which exceeds the critical value required to reject the null hypothesis. SPSS determined the exact p value to be .02. This p value is less than .05, which indicates that the null hypothesis should be rejected for the alternative hypothesis (that is, the two groups are significantly different in mean OCB). Masters Level Quantitative Design Assessment T-Scores

Effect Size

A common index of effect size for the independent samples t-test is eta squared (η2). SPSS does not provide this output for the independent samples t-test, but it is easily calculated by hand with the following formula: t2 ÷ (t2 + df). In the example above, the calculation is (-2.58)2 ÷ [(-2.58)2 + 16] = 6.65 ÷ (6.65 + 16) = 6.65 ÷ 22.65 = .29. This eta squared value falls between < .20 and > .80, and is therefore a “medium” effect size.

References

Lane, D. M. (2013). HyperStat online statistics textbook. Retrieved from http://davidmlane.com/hyperstat/index.html

Warner, R. M. (2013). Applied statistics: From bivariate through multivariate techniques (2nd ed.). Thousand Oaks, CA: Sage Publications.

 

DATA ANALYSIS AND APPLICATION TEMPLATE 1

DATA ANALYSIS AND APPLICATION TEMPLATE 4

Data Analysis and Application (DAA) Template

Learner Name

Capella University 

Data Analysis and Application (DAA) Template

Use this file for all assignments that require the DAA Template. Although the statistical tests will change from week to week, the basic organization and structure of the DAA remains the same. Update the title of the template. Remove this text and provide a brief introduction.

Section 1: Data File Description

Describe the context of the data set. You may cite your previous description if the same data set is used from a previous assignment.

Specify the variables used in this DAA and the scale of measurement of each variable.

Specify sample size (N).

Section 2: Testing Assumptions

1. Articulate the assumptions of the statistical test.

Paste SPSS output that tests those assumptions and interpret them. Properly integrate SPSS output where appropriate. Do not string all output together at the beginning of the section.

Summarize whether or not the assumptions are met. If assumptions are not met, discuss how to ameliorate violations of the assumptions.

Section 3: Research Question, Hypotheses, and Alpha Level

1. Articulate a research question relevant to the statistical test.

2. Articulate the null hypothesis and alternative hypothesis.

3. Specify the alpha level.

Section 4: Interpretation

1. Paste SPSS output for an inferential statistic. Properly integrate SPSS output where appropriate. Do not string all output together at the beginning of the section.

2. Report the test statistics.

3. Interpret statistical results against the null hypothesis.

Section 5: Conclusion

1. State your conclusions.

2. Analyze strengths and limitations of the statistical test.

 

References

Provide references if necessary.

SPSS Data Analysis Report Guidelines

For the SPSS data analysis report assignments in Assessments 2, 3, and 4, you will use the Data Analysis and Application (DAA) Template with the five sections described below. As shown in the IBM SPSS step-by-step guides, label all tables and graphs in a manner consistent with Capella’s APA Style and Format guidelines. Citations, if needed, should be included in the text and references included in a reference section at the end of the report. The organization of the report should include the following five sections:

Section 1: Data File Description (One Paragraph)

1. Describe the context of the data set. Cite a previous description if the same data set is used from a previous assignment. To increase the formal tone of the DAA, avoid first-person perspective “I.” For example, do not write, “I ran a scatter plot shown in Figure 1.” Instead, write, “Figure 1 shows. . . .”

2. Specify the variables used in this DAA and the scale of measurement of each variable.

3. Specify sample size (N).

Section 2: Testing Assumptions (Multiple Paragraphs)

1. Articulate the assumptions of the statistical test.

2. Paste SPSS output that tests those assumptions and interpret them. Properly embed SPSS output where appropriate. Do not string all output together at the beginning of the section. In other words, interpretations of figures and tables should be near (that is, immediately above or below) where the output appears. Format figures and tables per APA formatting. Refer to the examples in the IBM SPSS step-by-step guides.

3. Summarize whether or not the assumptions are met. If assumptions are not met, discuss how to ameliorate violations of the assumptions.

Section 3: Research Question, Hypotheses, and Alpha Level (One Paragraph)

1. Articulate a research question relevant to the statistical test.

2. Articulate the null hypothesis and alternative hypothesis for the research question.

3. Specify the alpha level (.05 unless otherwise specified).

Section 4: Interpretation (Multiple Paragraphs)

1. Paste SPSS output for an inferential statistic and report it. Properly embed SPSS output where appropriate. Do not string all output together at the beginning of the section. In other words, interpretations of figures and tables should be near (that is, immediately above or below) where the output appears. Format figures and tables per APA formatting. Masters Level Quantitative Design Assessment T-Scores

2. Report the test statistics. For guidance, refer to the “Results” examples at the end of the appropriate chapter of your Warner text.

3. Interpret statistical results against the null hypothesis.

Section 5: Conclusion (Two Paragraphs)

1. Provide a brief summary (one paragraph) of the DAA conclusions.

2. Analyze strengths and limitations of the statistical test.

Scoring Guide Tool

Print

PSY-FP7864 – Section 02

[u03a1] – t Test

Moore , Ashley

Use the following scoring guide, along with the project information, for evaluating each learner’s work.

Assessment InstructionsRead Assessment 3 Context (linked in the Resources) to learn about the concepts used in this assessment.

You will use the following resources for this assessment. They are linked in the Resources.

• Complete this assessment using the DAA Template.
• Read the SPSS Data Analysis Report Guidelines for a more complete understanding of the DAA Template and how to format and organize your assessment.
• Refer to IBM SPSS Step-By-Step Instructions: t Tests for additional information on using SPSS for this assessment.
• If necessary, review the Copy/Export Output Instructions to refresh your memory on how to perform these tasks. As with your previous assessments, your submission should be narrative with supporting statistical output (table and graphs) integrated into the narrative in the appropriate place (not all at the end of the document).

You will analyze the following variables in the grades.sav data set:

• gender.
• gpa.

Step 1: Write Section 1 of the DAA

• Provide a context of the grades.sav data set.
• Include a definition of the specified variables (predictor, outcome) and corresponding scales of measurement.
• Specify the sample size of the data set.

Step 2: Write Section 2 of the DAA

• Analyze the assumptions of the t test.
• Paste the SPSS histogram output for gpa and discuss your visual interpretations.
• Paste SPSS descriptives output showing skewness and kurtosis values for gpa and interpret them.
• Paste SPSS output for the Shapiro-Wilk test of gpa and interpret it.
• Report the results of the Levene’s test and interpret it.
• Summarize whether or not the assumptions of the t test are met.

Step 3: Write Section 3 of the DAA

• Specify a research question related to gender and gpa.
• Articulate the null hypothesis and alternative hypothesis.
• Specify the alpha level.

Step 4: Write Section 4 of the DAA

• Paste the SPSS output of the t test.
• Report the results of the SPSS output using proper APA guidelines. Include the following:
  • t.
  • Degrees of freedom.
  • t value.
  • p value.
  • Effect size.
  • Interpretation of effect size.
  • Means and standard deviations for each group. Mean difference.
  • Interpret the results against the null hypothesis.

Step 5: Write Section 5 of the DAA

• Discuss the implications of this t test as it relates to the research question.
• Conclude with an analysis of the strengths and limitations of t-test analysis.

 

Overall Comments

 

Ashley, you did a great job using SPSS to analyze the data and the results were interpreted correctly.  The research question and hypotheses were not quite right.  The t-test is not a correlational statistic, but Section 1 and Section 3 talk about it like it is.  Please see my comments and let me know if you have any questions.

Lorie

Criterion

Non-performance

Basic

Proficient

Distinguished

Criterion

Apply the appropriate SPSS procedures to check assumptions and calculate the independent-samples t test to generate relevant output.

(14%) CompetencyApply a statistical program’s procedure to data.

not selected Does not provide SPSS output. not selected Provides SPSS output with errors. not selected Applies the appropriate SPSS procedures to check assumptions and calculate the independent-samples t test to generate relevant output. selected Distinguished Analyzes the SPSS procedures to check assumptions and calculate the independent-samples t test to generate relevant output.

Comments:

Great job using SPSS to analyze the data with an ANOVA!

Criterion

Develop a context for the data set, including a definition of required variables and scales of measurement.

(14%) CompetencyApply the results of statistical analyses (your own or others) to your field of interest or career.

not selected Does not identify a context for the data set. selected Basic Identifies a context for the data set. not selected Develops a context for the data set, including a definition of required variables and scales of measurement. not selected Analyzes the context for the data set, demonstrating insight and understanding of relevant data required variables and scales of measurement.

Comments:

You say that the variables included in this assessment are GPA, total, ethnicity, final and gender. Only two of those were supposed to be used for this assessment. When you say a variable is scale you need to identify which scale. There shouldn’t be a discussion about correlations in this assessment.

Criterion

Analyze the assumptions of the independent-samples t test.

(14%) CompetencyAnalyze the decision-making process of data analysis.

not selected Does not identify the assumptions of the independent-samples t test. not selected Identifies the assumptions of the independent-samples t test. selected Proficient Analyzes the assumptions of the independent-samples t test. not selected Evaluates the assumptions of the independent-samples t test. Masters Level Quantitative Design Assessment T-Scores

Comments:

Good job! You say skew is not statistically significant. There aren’t statistics that tell if the skew or kurtosis is statistically significant. You can only say if the values fall within the acceptable range.

Criterion

Develop a research question, null hypothesis, alternative hypothesis, and alpha level.

(14%) CompetencyApply knowledge of hypothesis testing.

not selected Does not develop a research question, null hypothesis, alternative hypothesis, and alpha level. selected Basic Develops a research question, but does not include a null hypothesis, alternative hypothesis, or alpha level. not selected Develops a research question, null hypothesis, alternative hypothesis, and alpha level. not selected Develops a research question, null hypothesis, alternative hypothesis, and alpha level, demonstrating insight and understanding of the relevant data.

Comments:

You say, “We would want to establish if gender can explain the gpa; that is to check for correlation between the two variables.” The t-test is not a correlational test. In t-tests, we aren’t looking for relationships between variables. We are looking for differences between groups, and the research question and hypotheses should reflect that.

Criterion

Interpret the output of the independent-samples t test.

(14%) CompetencyInterpret the results of statistical analyses.

not selected Does not interpret the output of the independent-samples t test. not selected Generates, but does not interpret, the output of the independent-samples t test. not selected Interprets the output of the independent-samples t test. selected Distinguished Analyzes the independent-samples t test output.

Comments:

Great job interpreting the results of the t-test!

Criterion

Develop a conclusion that includes strengths and limitations of an independent-samples t test.

(15%) CompetencyAnalyze the computation, application, strengths, and limitations of various statistical tests.

not selected Does not develop a conclusion that includes the strengths and limitations of an independent-samples t test. not selected Develops a conclusion that does not include, or only partially includes, strengths and limitations of the independent-samples t test. not selected Develops a conclusion that includes strengths and limitations of an independent-samples t test. selected Distinguished Analyzes the strengths and limitations of an independent-samples t test, demonstrating insight and understanding of the relevant data. Masters Level Quantitative Design Assessment T-Scores

Comments:

Good job!

Criterion

Communicate in a manner that is scholarly, professional, and consistent with expectations for members of the identified field of study.

(15%) CompetencyCommunicate in a manner that is scholarly, professional, and consistent with the expectations for members in the identified field of study.

not selected Does not communicate in a manner that is scholarly, professional, or consistent with expectations for members of the identified field of study. not selected Inconsistently communicates in a manner that is scholarly, professional, and consistent with expectations for members of the identified field of study. not selected Communicates in a manner that is scholarly, professional, and consistent with expectations for members of the identified field of study. selected Distinguished Communicates in a manner that is scholarly, professional, and consistent with expectations for members of the identified field of study. Adheres to APA guidelines, and work is appropriate for publication. Masters Level Quantitative Design Assessment T-Scores

Comments:

This assessment was well-written.