Paired sample t-test in R

See Video ⮞ ☝

AGRON stats

June 27, 2019

• Introduction
• Formulation of hypothesis
• Example data set
• Descriptive analysis
  • Association or correlation test
  • Visualizing samples
  • Homogeneity in variances
• Paired sample t-test
• Interpretation

Introduction

A paired-samples t-test evaluates the mean within-subject difference between two measurements. Each “before” value must be matched to the corresponding “after” value for the same participant or experimental unit. The analysis is conducted on the paired differences, not on two independent samples.

The key assumptions are that pairs are independent of other pairs, the outcome is quantitative, and the distribution of within-pair differences is approximately normal when the sample is small. Inspect the differences and potential outliers; equal variances between the two time points is not an assumption of the paired t-test.

• The dependent variable must be numeric and continuous
• The observations are independent of one another
• The dependent variable should be approximately normally distributed
• The dependent variable should not contain any outliers

We will use a training example to demonstrate how to prepare paired data, inspect the differences, and run the test in R. Define the direction of subtraction before interpreting the estimated mean change.

Suppose ICT knowledge scores were recorded for the same participants before and after training. A paired analysis is appropriate because observations are linked within participant; the data should not be treated as independent pre-training and post-training groups.

Formulation of hypothesis

The example asks whether the training changes the mean score at a two-sided 5% significance level. If the practical question specifically concerns improvement, define the direction in advance and use a one-sided test only when that direction was prespecified.

• The null hypothesis states that the population mean paired difference is zero. The alternative states that it is nonzero for a two-sided test. Choose the alternative before examining the results.

• H0: the population mean change in ICT score is zero.

H1: the population mean change in ICT score is not zero for the two-sided test.

Note: It is important to remember that hypotheses are never about data, they are about the processes which produce the data.

Example data set

A directional alternative can be used when the research question and analysis plan prespecify the expected direction. A one-sided test does not simply increase power without consequence: it cannot detect an effect in the opposite direction as evidence for the stated alternative.

The example uses ten participants measured at two time points. The table should contain one record per participant and time, with a consistent participant identifier so each pair can be matched.

The example scores are shown below. Check that each participant has exactly one measurement at each time point before reshaping or testing.

before <- c(12.2, 14.6, 13.4, 11.2, 12.7, 10.4, 15.8, 13.9, 9.5, 14.2)
after <- c(13.5, 15.2, 13.6, 12.8, 13.7, 11.3, 16.5, 13.4, 8.7, 14.6)

Now create data matrix using data.frame() function. In this function you can define the variables. First variable is subject or individuals. As sample size is \(10\) so we shall repeat \(1\) to \(10\) twice. Second variable is time interval containing two levels. Each level will be repeated \(10\) times. Third variable is ICT score representing response variable values. Use objects before and after as created earlier to represent scores. Let’s print the data to see how it looks like.

data <- data.frame(subject = rep(c(1:10), 2), 
                   time = rep(c("before", "after"), each = 10),
                   score = c(before, after))
print(data)

#    subject   time score
# 1        1 before  12.2
# 2        2 before  14.6
# 3        3 before  13.4
# 4        4 before  11.2
# 5        5 before  12.7
# 6        6 before  10.4
# 7        7 before  15.8
# 8        8 before  13.9
# 9        9 before   9.5
# 10      10 before  14.2
# 11       1  after  13.5
# 12       2  after  15.2
# 13       3  after  13.6
# 14       4  after  12.8
# 15       5  after  13.7
# 16       6  after  11.3
# 17       7  after  16.5
# 18       8  after  13.4
# 19       9  after   8.7
# 20      10  after  14.6

Create vectors for the before- and after-training scores in matching participant order. Confirm that the same index refers to the same individual in both vectors; otherwise the paired analysis will be invalid.

str(data)

# 'data.frame': 20 obs. of  3 variables:
#  $ subject: int  1 2 3 4 5 6 7 8 9 10 ...
#  $ time   : Factor w/ 2 levels "after","before": 2 2 2 2 2 2 2 2 2 2 ...
#  $ score  : num  12.2 14.6 13.4 11.2 12.7 10.4 15.8 13.9 9.5 14.2 ...

attach(data)

Descriptive analysis

Build a data frame with participant ID, time point, and score. Each participant should appear once for each time, and the levels of the time factor should be ordered clearly for interpretation.

summary(data)

#     subject         time        score      
#  Min.   : 1.0   after :10   Min.   : 8.70  
#  1st Qu.: 3.0   before:10   1st Qu.:11.97  
#  Median : 5.5               Median :13.45  
#  Mean   : 5.5               Mean   :13.06  
#  3rd Qu.: 8.0               3rd Qu.:14.30  
#  Max.   :10.0               Max.   :16.50

Use str() to confirm the ID, time, and score types. The example uses attach(); explicit references such as data$score are safer and clearer in reproducible scripts.

by(data = data, 
   INDICES = data[,"time"], 
   FUN = summary)

# data[, "time"]: after
#     subject          time        score      
#  Min.   : 1.00   after :10   Min.   : 8.70  
#  1st Qu.: 3.25   before: 0   1st Qu.:12.95  
#  Median : 5.50               Median :13.55  
#  Mean   : 5.50               Mean   :13.33  
#  3rd Qu.: 7.75               3rd Qu.:14.38  
#  Max.   :10.00               Max.   :16.50  
# ------------------------------------------------------------ 
# data[, "time"]: before
#     subject          time        score      
#  Min.   : 1.00   after : 0   Min.   : 9.50  
#  1st Qu.: 3.25   before:10   1st Qu.:11.45  
#  Median : 5.50               Median :13.05  
#  Mean   : 5.50               Mean   :12.79  
#  3rd Qu.: 7.75               3rd Qu.:14.12  
#  Max.   :10.00               Max.   :15.80

Inspect the score distributions and summarize each time point before testing. Descriptive statistics help identify coding errors and provide context for the estimated change.

diff = after - before
summary(diff)

#    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
#  -0.800   0.250   0.650   0.540   0.975   1.600

Association or correlation test

Calculate summaries by time point to compare the observed score distributions. The same participants contribute measurements at both times, so describe the data as paired.

library(stats)
cor.test(x = before, y = after, 
         method = c("pearson"), 
         conf.level = 0.95)

# 
#   Pearson's product-moment correlation
# 
# data:  before and after
# t = 7.5468, df = 8, p-value = 6.628e-05
# alternative hypothesis: true correlation is not equal to 0
# 95 percent confidence interval:
#  0.7474506 0.9851801
# sample estimates:
#       cor 
# 0.9363955

Compute the paired difference using a stated order, for example after minus before. A positive mean difference then corresponds to an increase after training; keep this convention consistent in the test and report.

Visualizing samples

A correlation between paired measurements can describe whether participants with higher baseline scores also tend to have higher follow-up scores. It is not an assumption test for the paired t-test and does not test whether the training changed the mean.

par(pty = "s")
boxplot(score ~ time)
boxplot(score ~ time, 
        col = c("#003C67FF", "#EFC000FF"),
        main = "ICT training score improves knowlege",
        xlab = "Time", ylab = "Score")

The reported correlation of 0.94 indicates a strong positive linear association between the two sets of scores in this sample. It does not establish agreement between measurements or demonstrate a training effect.

Plot the scores by time and, preferably, connect each participant’s paired observations. A boxplot can summarize marginal distributions, but it hides the within-person changes that drive the paired test.

• Upper and lower whisker represents the scores outside the middle 50 percent
• Upper and lower quartiles
• 75 percent of the scores fall below the upper quartile and 25% of the scores fall below the lower quartile
• The middle box represents the middle 50 percent of the score (Interquartile range)
• The median or middle quartile represent the midpoint of the data

A boxplot summarizes the median, quartiles, and potential outlying values under its plotting rule. For paired data, add paired points or lines so readers can see each participant’s change.

Homogeneity in variances

Before proceeding for test statistics, first determine the homogeneity of variances of time variable for ICT training scores. You can use bartlett.test() where score is separated by time variable.

bartlett.test(score ~ time)

# 
#   Bartlett test of homogeneity of variances
# 
# data:  score by time
# Bartlett's K-squared = 0.050609, df = 1, p-value = 0.822

Bartlett’s test compares group variances under assumptions that include normality. It is not required for a paired t-test, which analyzes within-pair differences; do not use this test to decide whether to set a paired analysis option.

Paired sample t-test

A non-significant Bartlett test does not prove that variances are equal. In any event, equality of pre- and post-training variances is not a condition for the paired t-test.

t.test(formula = score ~ time,
       alternative = "greater",
       mu = 0, 
       paired = TRUE,   
       var.equal = TRUE,
       conf.level = 0.95)

# 
#   Paired t-test
# 
# data:  score by time
# t = 2.272, df = 9, p-value = 0.0246
# alternative hypothesis: true difference in means is greater than 0
# 95 percent confidence interval:
#  0.1043169       Inf
# sample estimates:
# mean of the differences 
#                    0.54

Interpretation

Use t.test() with the two measurements paired by participant, a null mean difference of zero, the prespecified alternative, and the desired confidence level. The order of the two samples determines the sign of the estimated difference. For a paired test, var.equal is not relevant; do not interpret it as a variance-homogeneity adjustment.

The p-value is the probability, assuming the null hypothesis and test assumptions, of obtaining a test statistic at least as extreme as the observed one under the specified alternative. It is not the probability that the null hypothesis is true or the probability that the result occurred “by chance.”

Please comment below if you have any questions.

---

Compare the p-value with the prespecified significance level and report the estimated mean paired difference with its confidence interval. If the interval and estimate support an increase, describe the result as an observed change in this sample; causal claims about training effectiveness require a design that rules out alternative explanations.

---

Download R program — Click_here

Download R studio — Click_here

---