Statistical Methods with R

Populations & Samples

Unit B · Chapter 04 · Lecture 04a

Developed by Jeffrey M. Girard

Roadmap: Sampling & Estimation

  1. Research Methods

  2. Populations and Parameters

  3. Samples and Statistics

  4. Estimation and Uncertainty

Research Methods

The scientific method

  • Uncertainty can be scary, uncomfortable, and dangerous

  • We strive to understand the world and make predictions

  • We often use our intuition and experiences to do so

    • Intuitions and experience are usually better than nothing
    • However, they can often be wrong and misleading!

  • We can do much better using the scientific method

See The Demon-Haunted World: Science as a Candle in the Dark by Sagan & Druyan

What is the scientific method?

  • Start with a research question that is answerable with data

  • Generate a hypothesis (i.e., a testable prediction)

    • Hypotheses usually derive from scientific theories
    • Hypotheses need to be operationalized and falsifiable

  • Conduct a replicable study to collect relevant data

  • Analyze the data to test the hypothesis/predictions

The scientific method classically uses experimental studies, but it can also improve observational studies

Example

  • Question: Does caffeine make you smarter?

  • Hypothesis: Adult humans given caffeine will navigate a maze faster than adult humans not given caffeine.

  • Experiment: Recruit adult humans and give one group decaffeinated coffee and give another group normal coffee. Measure the time it takes each human to navigate a maze.

  • Analysis: Compare the two groups’ maze completion times.

What role do statistics play?

  • Descriptive Statistics summarize and visualize the distributions and associations between variables

  • Inferential Statistics allow us to make inferences about our data and quantify the uncertainty in those inferences

    • Probability: Model \to Data
      (e.g., given a fair coin, what is the chance of HHHHH…)

    • Statistics: Data \to Model
      (e.g., given HHHHH, what is the chance the coin is fair…)

Populations and Parameters

What is a population?

  • The population is the set of all possible observations that we want to draw conclusions and make predictions about

  • Imagine that we are interested in studying sleeping behavior

  • Are we interested in understanding the behavior of…

    • All humans who ever lived and ever will live
    • All adults (18 years and older) living in the USA
    • All students enrolled in this course this semester

Populations aren’t always made up of people; you could have populations of groups, objects, events, etc.

How large is the population?

  • The number of individuals in a population is denoted N
    • The number of humans in past and future, N\rightarrow\infty
    • The number of adults in the USA, N\approx209 million
    • The number of students in this class, N=24
  • We need to be very careful about generalizing/extrapolating
    • Conclusions may not apply to other populations
    • Predictions may be less accurate in other populations

What is a parameter?

  • Parameters are things we want to learn about a population
    • The typical sleep duration for students
    • The spread in sleep duration across students
  • And beyond (e.g., other summaries, differences, associations)

  • A few years ago, I sent out a survey to every student in class

  • I was able to gather data for this entire population!

Small population example

# Read in data from file
library(tidyverse)
sleep <- read_csv("sleep.csv")

# Calculate total sleep hours
sleep$total <- 
  sleep$night + sleep$day
sleep$total
 [1] 8.0 6.0 7.5 9.0 9.0 8.0 6.0 8.0 7.0 9.0 8.0 9.0 6.0 7.5 7.0 7.0
qplot(x = total, data = sleep, 
      geom = "density")

Small population example

Population Mean

The mean represents the “typical” sleep duration

\mu=\frac{1}{N} \sum_{i=1}^N x_i

x <- sleep$total
N <- length(x)
mu <- (1 / N) * sum(x)
mu
## [1] 7.625

Population Std. Deviation

The SD represents the “spread” across individuals

\sigma=\sqrt{\frac{1}{N}\sum_{i=1}^{N}(x_i-\mu)^2}

sqdiff <- (x - mu)^2
sigma <- sqrt((1 / N) * sum(sqdiff))
sigma
## [1] 1.038328

We denote parameters using Greek letters (\mu,\sigma) or uppercase English letters (N)

Samples and Statistics

Sampling

  • In practice, we are often interested in very large populations

    • N = 209M adults in USA, N = 35T blood cells in body

  • It would be infeasible to measure such populations entirely

  • In these cases, we rely on sampling and statistical inference

  • A sample is a subset drawn from a population

    • n = 600 adults from USA, n = 5M blood cells from body

Note we use lowercase n to denote the sample size

Basics of statistical inference

  • Sampling draws a subset to represent a population

  • Inference is learning about the population by studying a sample drawn from it

  • We measure in the sample what we want to know

  • This measure is the sample statistic

  • The sample statistic estimates the population parameter

Sample example

  • Our population is all the students in the class (N=16) and our parameters of interest are the mean and SD of their total sleep time (\mu=7.63, \sigma=1.04)

  • But imagine it is not feasible or possible to survey the entire population… so we randomly select six students to survey

sample() randomly draws a sample (of given size) from a vector

Because sampling is random, we each get a different sample.

xs <- sample(sleep$total, size=6)
xs
## [1] 6.0 8.0 7.5 9.0 7.0 9.0

In practice, we usually only have the sample’s data and so using sample() isn’t usually necessary

Sample example

Sample Mean

The mean formulas are basically identical for both samples and populations

\bar{x}=\frac{1}{n} \sum_{i=1}^n x_i

xbar <- mean(xs)
xbar
## [1] 7.75

Sample Std. Deviation

But the SD formulas differ a little (we subtract 1 to correct for small-sample bias)

s=\sqrt{\frac{1}{\color{red}{n-1}}\sum_{i=1}^{n}(x_i-\bar{x})^2}

s <- sd(xs)
s
## [1] 1.172604

We denote sample statistics using lowercase English letters (n, \bar{x},s) or Greek letters with hats (\hat{\mu},\hat{\sigma})

Estimation and Uncertainty

Sampling error

  • Uh-oh, our statistics \neq their corresponding parameters!
round(c(mu, xbar, xbar - mu), 2)
## [1] 7.62 7.75 0.12
round(c(sigma, s, s - sigma), 2)
## [1] 1.04 1.17 0.13
  • Such differences are called sampling errors
  • Sampling errors come from two main sources
    • Stochastic Error: The sample was too small to be perfectly representative of the population
    • Systematic Error: The sample was not random enough to be perfectly representative of the population

Why does sample size matter?

  • Imagine we had drawn a sample of 2 instead of 6…

  • We might only sample part of the distribution

    • The sample mean would depend on who was included
    • We might end up with only extreme people
    • Our estimate would thus be more influenced by chance

  • Now imagine we had drawn a sample of 14 instead of 6…

    • The sample mean wouldn’t depend on who was included
    • Our estimate would thus be less influenced by chance

The law of large numbers

  • Larger random samples approximate the population better

  • Statistics from larger (random) samples will tend to better estimate the parameters

Small population simulation

Let’s draw samples of different sizes and calculate the sample mean

To show chance at work, we repeat this 50 times per sample size

Let’s plot each repetition’s sample mean as a point (and the population mean as a line)

Large population simulation

Do much larger populations require much larger samples? Let’s find out…

Here is a simulation of sampling from a population N=1000000, \mu=100, \sigma=15

Insights from the simulations

  1. Repetitions of a study (with different samples) will result in different estimates due to sampling error

  2. There is thus some uncertainty in our estimates/inferences due to sampling error

  3. We can quantify the amount of sampling error in an estimate

  4. Larger samples tend to decrease this uncertainty and “stabilize” our estimates (reducing sampling error)

  5. Perhaps surprisingly, this depends on the sample’s size, not on the fraction of the population sampled

Sampling distributions

  • The sampling distribution contains the statistic values from repeated samples

  • The wider the sampling distribution, the more uncertainty there is

  • Its SD has its own name: the standard error

Standard errors

We can rarely take repeated samples, build the sampling distribution, and read the SE off it

Instead we estimate the SE from the data, just like a parameter

The estimate for the SE of the mean depends on two main things:

  1. How large the sample is (n)

  2. How spread out the data is (s)

Standard Error of the Mean

s_{\bar{x}} = \frac{s}{\sqrt{n}} \sqrt{\frac{N-n}{N-1}}

As s increases, the SE increases

As n increases, the SE decreases

As n\to N, the second part \to zero

As N\to\infty, the second part \to one

Because we are often interested in infinite populations, the right side of this equation is often left off entirely.

Example

Smaller Sample (n=3)

n <- 3
xs <- sample(sleep$total, n)
N <- length(sleep$total)
s <- sd(xs)
se_m <- (s / sqrt(n)) * 
  sqrt((N - n)/(N - 1))
se_m
## [1] 0.2687419

Larger Sample (n=12)

n <- 12
xs <- sample(sleep$total, n)
N <- length(sleep$total)
s <- sd(xs)
se_m <- (s / sqrt(n)) * 
  sqrt((N - n)/(N - 1))
se_m
## [1] 0.1456438

The larger sample thus has a smaller estimated SE