BIOS-338/538 - lecture 17: t-test and bootstrapping

Dataset for t-testing

Many flavours of t-tests

1-sample vs 2-sample
- >2 samples ~ do ANOVA
1-tailed vs 2-tailed
Paired vs unpaired

Visualize before 2-sample t-test

Here’s a plot outlining the data to use for t-test

2-sample t-test in R

# Show code and do t-test for vericolor vs virginica
t.test(Sepal.Length ~ Species, data = iris2)


    Welch Two Sample t-test

data:  Sepal.Length by Species
t = -5.6292, df = 94.025, p-value = 1.866e-07
alternative hypothesis: true difference in means between group versicolor and group virginica is not equal to 0
95 percent confidence interval:
 -0.8819731 -0.4220269
sample estimates:
mean in group versicolor  mean in group virginica 
                   5.936                    6.588

What is p-value?

p-value is the probability that the observed difference of means (or more extreme) can occur by chance if the NULL hypothesis is TRUE

This is calculated by

plotting a t-distribution around the null hypothesis mean difference (typically 0)
mark the observed mean difference
Find the tail of the distribution beyond the observed value

Bootstrapping to understand t-test better

Bootstrapping shows us the variability around the mean, by virtually repeating the experiment a bunch of times

For 1 sample, this is how the bootstrapped distribution looks like

Code

set.seed(1)
sample_1 <- runif(n = 10, -1, 1.5) # make dataset
sample_1 %>% sort() %>% round(2) %>% print # show dataset

 [1] -0.85 -0.50 -0.34 -0.07  0.43  0.57  0.65  1.25  1.27  1.36

1-sample t-test

Null hypothesis ( $H_0$ ) ~ [ $\mu = \mu_0$ ] => the mean of the data is $\mu_0$

We will test for: $\mu = 0$ for this dataset

t_test_1 <- t.test(sample_1, mu = 0) %>% print


    One Sample t-test

data:  sample_1
t = 1.5179, df = 9, p-value = 0.1634
alternative hypothesis: true mean is not equal to 0
95 percent confidence interval:
 -0.1857309  0.9433002
sample estimates:
mean of x 
0.3787847

t_tst_p <- t_test_1$p.value %>% round(2)
str_c('p-value for t-test is: ', t_tst_p)

[1] "p-value for t-test is: 0.16"

1-sample t-test w bootstrapping

To get p-value from the bootstrapping, we need to find the area of the tails. To facilitate understanding, we shift the distribution to fit the null hypothesis. Which means, the mean of the distribution should be moved to the null hypothesis!

Showing the 2 tails

For a 2 tailed test, the p-value corresponds to the area under these two tails

Tail 1: $\mu > \mu_0$
Tail 2: $\mu < \mu_0$
Both tails: $\mu != \mu_0$

Calculating the p-value from bootstrap dist.

Graphically area is easy to visualize,

For calculation, probability is just the number of values in the tails / total number of values

first_tail <- sum(boot_1 - bootmean_1 > bootmean_1)
second_tail <- sum(boot_1 < 0)  

boot_p_val <- (first_tail + second_tail)/length(boot_1)

str_c('p-value for bootstrapping t-test: ', boot_p_val)

[1] "p-value for bootstrapping t-test: 0.1067"

str_c('p-value for t-test is: ', t_tst_p)

[1] "p-value for t-test is: 0.16"

p-value for t-test is: 0.16

p-value for t-test is: 0.1067

Exploring SEM, CI within bootstrap dist.

(SEM) Standard error of mean = std deviation of the bootstrapping distribution
(CI) 95% confidence interval => 95% of the mean distribution area lies within this range

Onto the worksheet : 2 samples t-test

Please download/git pull the class17_t-test_bootstrapping.qmd worksheet for today from Github

Use the viridis dataset : virginica vs versicolor
- [second section: Mutate new columns to generate: versicolor (+ 0.5)]
Do t.test and record the p-value
Do bootstrapping with map
Plot histogram, red line with null hypothesis
Calculate p value from bootstrap hist
Show side by side comparison of both p-values
Use automated moderndive:: functions to get the bootstrapped p-value?

lecture 17: t-test and bootstrapping

Recap

Reminder on hypothesis testing

Difference of mean between samples

Sampling from a population

Re-sampling = shuffling within the sample (w replacement)

Reminder: Recipe for simulation/lec12

Today’s class