Statistical Methods with R

Populations & Samples

Unit B · Chapter 04 · Lecture 04a

Developed by Jeffrey M. Girard

Research Methods

The scientific method

  • Uncertainty can be scary, uncomfortable, and dangerous

  • We strive to understand the world and make predictions

  • We often use our intuition and experiences to do so

    • Intuitions and experience are usually better than nothing
    • However, they can often be wrong and misleading!

  • We can do much better using the scientific method

See The Demon-Haunted World: Science as a Candle in the Dark by Sagan & Druyan

What is the scientific method?

  • Start with a research question that is answerable with data

  • Generate a hypothesis (i.e., a testable prediction)

    • Hypotheses usually derive from scientific theories
    • Hypotheses need to be operationalized and falsifiable

  • Conduct a replicable study to collect relevant data

  • Analyze the data to test the hypothesis/predictions

The scientific method classically uses experimental studies, but it can also improve observational studies

Example

  • Question: Does caffeine make you smarter?

  • Hypothesis: Adult humans given caffeine will navigate a maze faster than adult humans not given caffeine.

  • Experiment: Recruit adult humans and give one group decaffeinated coffee and give another group normal coffee. Measure the time it takes each human to navigate a maze.

  • Analysis: Compare the two groups’ maze completion times.

What role do statistics play?

  • Descriptive Statistics summarize and visualize the distributions and associations between variables

  • Inferential Statistics allow us to make inferences about our data and quantify the uncertainty in those inferences

    • Probability: Model \(\to\) Data
      (e.g., given a fair coin, what is the chance of HHHHH…)

    • Statistics: Data \(\to\) Model
      (e.g., given HHHHH, what is the chance the coin is fair…)

Populations and Parameters

What is a population?

  • The population is the set of all possible observations that we want to draw conclusions and make predictions about

  • Imagine that we are interested in studying sleeping behavior

  • Are we interested in understanding the behavior of…

    • All humans who ever lived and ever will live
    • All adults (18 years and older) living in the USA
    • All students enrolled in this course this semester

Populations aren’t always made up of people; you could have populations of groups, objects, events, etc.

How large is the population?

  • The number of individuals in a population is denoted \(N\)
    • The number of humans in past and future, \(N\rightarrow\infty\)
    • The number of adults in the USA, \(N\approx209\) million
    • The number of students in this class, \(N=24\)
  • We need to be very careful about generalizing/extrapolating
    • Conclusions may not apply to other populations
    • Predictions may be less accurate in other populations

What is a parameter?

  • Parameters are things we want to learn about a population
    • The typical sleep duration for students
    • The spread in sleep duration across students
  • And beyond (e.g., other summaries, differences, associations)

  • A few years ago, I sent out a survey to every student in class

  • I was able to gather data for this entire population!

Small population example

# Read in data from file
library(tidyverse)
sleep <- read_csv("../../data/sleep.csv")

# Calculate total sleep hours
sleep$total <- 
  sleep$night + sleep$day
sleep$total
 [1] 8.0 6.0 7.5 9.0 9.0 8.0 6.0 8.0 7.0 9.0 8.0 9.0 6.0 7.5 7.0 7.0
qplot(x = total, data = sleep, 
      geom = "density")

Small population example

Population Mean

The mean represents the “typical” sleep duration

\[\mu=\frac{1}{N} \sum_{i=1}^N x_i\]

x <- sleep$total
N <- length(x)
mu <- (1 / N) * sum(x)
mu
## [1] 7.625

Population Std. Deviation

The SD represents the “spread” across individuals

\[\sigma=\sqrt{\frac{1}{N}\sum_{i=1}^{N}(x_i-\mu)^2}\]

sqdiff <- (x - mu)^2
sigma <- sqrt((1 / N) * sum(sqdiff))
sigma
## [1] 1.038328

We denote parameters using Greek letters \((\mu,\sigma)\) or uppercase English letters \((N)\)

Samples and Statistics

Sampling

  • In practice, we are often interested in very large populations

    • \(N\) = 209M adults in USA, \(N\) = 35T blood cells in body

  • It would be infeasible to measure such populations entirely

  • In these cases, we rely on sampling and statistical inference

  • A sample is a subset drawn from a population

    • \(n\) = 600 adults from USA, \(n\) = 5M blood cells from body

Note we use lowercase \(n\) to denote the sample size

Basics of statistical inference

  • Sampling is drawing a subset of observations to represent a population

  • Inference is learning about the population by studying a sample drawn from it

  • We first measure in the sample whatever we want to know about the population

  • We call this measure the sample statistic

  • We use the sample statistic to estimate (i.e., guess) the population parameter

Sample example

  • Our population is all the students in the class \((N=16)\) and our parameters of interest are the mean and SD of their total sleep time \((\mu=7.63, \sigma=1.04)\)

  • But imagine it is not feasible or possible to survey the entire population… so we randomly select six students to survey

The sample() function will randomly draw a sample (of given size) from a vector

Note that, because sampling is random, we will each get a slightly different sample.

xs <- sample(sleep$total, size=6)
xs
## [1] 6.0 8.0 7.5 9.0 7.0 9.0

In practice, we usually only have the sample’s data and so using sample() isn’t usually necessary

Sample example

Sample Mean

The mean formulas are basically identical for both samples and populations

\[\bar{x}=\frac{1}{n} \sum_{i=1}^n x_i\]

xbar <- mean(xs)
xbar
## [1] 7.75

Sample Std. Deviation

But the SD formulas are a bit different (we subtract 1 to correct for bias in small samples)

\[s=\sqrt{\frac{1}{\color{red}{n-1}}\sum_{i=1}^{n}(x_i-\bar{x})^2}\]

s <- sd(xs)
s
## [1] 1.172604

We denote sample statistics using lowercase English letters \((n, \bar{x},s)\) or Greek letters with hats \((\hat{\mu},\hat{\sigma})\)

Estimation and Uncertainty

Sampling error

  • Uh-oh, our statistics \(\neq\) their corresponding parameters!
round(c(mu, xbar, xbar - mu), 2)
## [1] 7.62 7.75 0.12
round(c(sigma, s, s - sigma), 2)
## [1] 1.04 1.17 0.13
  • Such differences are called sampling errors
  • Sampling errors come from two main sources
    • Stochastic Error: The sample was too small to be perfectly representative of the population
    • Systematic Error: The sample was not random enough to be perfectly representative of the population

Why does sample size matter?

  • Imagine we had drawn a sample of 2 instead of 6…

  • We might only sample part of the distribution

    • The sample mean would depend on who was included
    • We might end up with only extreme people
    • Our estimate would thus be more influenced by chance

  • Now imagine we had drawn a sample of 14 instead of 6…

    • The sample mean wouldn’t depend on who was included
    • Our estimate would thus be less influenced by chance

The law of large numbers

  • Larger (random) samples will tend to better approximate the population

  • Statistics from larger (random) samples will tend to better estimate the parameters

Small population simulation

Let’s take samples of different sizes from our population and calculate the sample mean

To show the influence of chance, we can repeat this 50 times for each sample size

Let’s plot each repetition’s sample mean as a point (and the population mean as a line)

Large population simulation

With larger populations, we need larger samples to get good estimates

Here is a simulation of sampling from a population \(N=1000000, \mu=100, \sigma=15\)

Insights from the simulations

  1. Repetitions of a study (with different samples) will result in different estimates due to sampling error

  2. There is thus some uncertainty in our estimates/inferences due to sampling error

  3. We can quantify the amount of sampling error in an estimate

  4. Larger samples tend to decrease this uncertainty and “stabilize” our estimates (reducing sampling error)

  5. However, larger populations require larger samples to achieve this benefit

Sampling distributions

  • The sampling distribution contains the statistic values from repeated samples

  • The wider the sampling distribution, the more uncertainty there is

  • The SD of the sampling distribution is called the standard error

Standard errors

We can rarely take many repeated samples, construct the sampling distribution, and calculate the SE directly

So, instead, we typically estimate the SE from the data (just like a parameter)

The estimate for the SE of the mean depends on two main things:

  1. How large the sample is \((n)\)

  2. How spread out the data is \((s)\)

Standard Error of the Mean

\[s_{\bar{x}} = \frac{s}{\sqrt{n}} \sqrt{\frac{N-n}{N-1}}\]

As \(s\) increases, the SE increases

As \(n\) increases, the SE decreases

As \(n\to N\), the second part \(\to\) zero

As \(N\to\infty\), the second part \(\to\) one

Because we are often interested in infinite populations, the right side of this equation is often left off entirely.

Example

Smaller Sample (n=3)

n <- 3
xs <- sample(sleep$total, n)
N <- length(sleep$total)
s <- sd(xs)
se_m <- (s / sqrt(n)) * 
  sqrt((N - n)/(N - 1))
se_m
## [1] 0.2687419

Larger Sample (n=12)

n <- 12
xs <- sample(sleep$total, n)
N <- length(sleep$total)
s <- sd(xs)
se_m <- (s / sqrt(n)) * 
  sqrt((N - n)/(N - 1))
se_m
## [1] 0.1456438

The larger sample thus has a smaller estimated SE