Statistical Methods with R

Confidence Intervals

Unit B · Chapter 04 · Lecture 04b

Developed by Jeffrey M. Girard

Roadmap: Sampling & Estimation

  1. Types of Estimates

  2. The Normal Distribution

  3. Central Limit Theorem

  4. Confidence Intervals

  5. Practical Matters

Types of Estimates

Point and Interval Estimates

In statistical inference, we will often want two types of estimates…

Point Estimates

  • What is our single best guess for the
    value of the population parameter?

  • This is usually the sample statistic

  • Says nothing about uncertainty

e.g., I don’t know the average height of all adult men, but I will guess it equals the average height in my sample

Interval Estimates

  • What is a range of reasonable guesses for the value of the population parameter?

  • Usually the statistic plus/minus an amount reflecting uncertainty

e.g., Given sampling error, any number within 2.9 cm of my sample average is also a reasonable guess

Reasonable values

  • Our uncertainty comes from the fact that sampling error causes different samples to yield different estimates

  • The sampling distribution shows what statistic values would be more or less common across many such samples

  • Reasonable values would be those where the sampling distribution is densest (i.e., the values that are the most commonly observed across different samples)

  • With less uncertainty (e.g., larger samples), the range of reasonable values will be tighter/narrower

Sampling distribution

  • But we don’t have access to the sampling distribution!
    • This is because we usually only take one sample…
  • So we will need to estimate the statistic values for which
    the sampling distribution is densest
    • There are many different techniques for doing this
  • We will use the frequentist parametric approach
    • This approach is very common and assumes that
      the sampling distribution is normally distributed
    • What does that mean? Is it reasonable to assume?

The Normal Distribution

The normal distribution

Also called the Bell Curve or the Gaussian distribution

We write “x is normal with mean \mu and SD \sigma” as

x \sim \text{Normal}(\mu, \sigma)\\x \sim \mathcal{N}(\mu,\sigma)

  • \mu is the mean, \sigma is the SD

\mathcal{N}(0,1) is the standard normal

The mu parameter

Changing \mu shifts the distribution left (-) or right (+)

The sigma parameter

Changing \sigma expands (+) or contracts (-) the distribution

The empirical rule

~68.3% of the Normal distribution falls within 1 SD of the mean
~95.4% of the Normal distribution falls within 2 SD of the mean
~99.7% of the Normal distribution falls within 3 SD of the mean

Quantiles in R

How many SDs contain a given percentage?

  • We want the middle 95%, so we cut the 5% in half

  • We thus calculate the 2.5th and 97.5th percentiles

  • qnorm() does this for the Normal distribution

qnorm(0.025)
## [1] -1.959964
qnorm(0.975)
## [1] 1.959964
  • So 95% of the Normal distribution is within ~1.96 SDs

Putting it all together

  1. Our interval should cover the densest part of the sampling distribution

  2. 95% of the density of a normal distribution is within ~1.96 SD of the mean

  3. The SD of the sampling distribution is the standard error (SE) of the statistic

  4. So, if we assume the sampling distribution is normally distributed, then…

  5. Our interval estimate is the sample statistic plus/minus 1.96 * SE

\text{CI}_{95} = \bar{x} \pm (1.96 \times s_{\bar{x}})

This is the 95% Confidence Interval (CI), and more on it soon

But this parametric approach all hinges on the assumption of normality (#4)…

Central Limit Theorem

Non-normally distributed data

What happens to the sampling distribution of \bar{x}
when x itself is not normally distributed?

Central limit theorem

The sampling distribution of \bar{x} will become more normal
as n increases (even if x itself is not normally distributed)

Normal approximations

  • The CLT applies to means and sums
    • And to statistics built from them
    • Why? A mean sums many independent observations
    • Some values will be higher and others will be lower
    • These deviations will tend to cancel each other out
    • Central values are common and extreme values are rarer
  • Thus, we often assume a normal sampling distribution…
    • This becomes more reasonable as n increases

Confidence Intervals

Calculations

  • To estimate the 95% Confidence Interval of the mean

\text{CI}_{95}= \bar{x}\pm\left(1.96\times\frac{\sigma}{\sqrt{n}}\right)

  • However, this requires that we know \sigma, the population SD

  • Usually we estimate \sigma using s, the sample SD

  • This estimation introduces a bit more uncertainty…

A complication

  • To account for this extra uncertainty, we need to…
    • Stop using the Normal distribution and qnorm()
    • Instead use the Student t distribution and qt()
  • The t distribution has more uncertainty when n is small
    • It becomes increasingly normal as n increases
    • The t distribution has the same \mu and \sigma parameters
    • But it also has “degrees of freedom” (\textit{df}=n-1)
    • As n increases, df increases, and t becomes more normal

The df parameter

As df increases, t(\textit{df},\mu,\sigma) becomes increasingly normal

Sometimes you’ll also see df referred to as \nu

Comparing multipliers

When n is small, the t multiplier is higher (more uncertainty)

As n increases, the multipliers become increasingly similar

A correction to our formula

Population SD is known

\text{CI}_{\%}=\bar{x}\pm z \left(\frac{\sigma}{\sqrt{n}}\right)

# n = 20
(z <- qnorm(0.975))
## [1] 1.959964

# n = 100
(z <- qnorm(0.975))
## [1] 1.959964

Population SD is estimated

\text{CI}_{\%}=\bar{x}\pm t \left(\frac{s}{\sqrt{n}}\right)

# n = 20
(t <- qt(0.975, df = 20 - 1))
## [1] 2.093024

# n = 100
(t <- qt(0.975, df = 100 - 1))
## [1] 1.984217

An example in R

tratings <- read_csv("teaching_ratings.csv")
summary(tratings$eval)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   2.300   3.647   3.935   3.906   4.215   4.790

Estimate mean and SE

x <- tratings$eval
n <- length(x)
xbar <- mean(x)
s <- sd(x)
se <- s / sqrt(n)

xbar
## [1] 3.906064
se
## [1] 0.04753455

Estimate confidence interval

t <- qt(0.975, df = n - 1)
ci_lo <- xbar - (t * se)
ci_hi <- xbar + (t * se)

ci_lo
## [1] 3.81167
ci_hi
## [1] 4.000458

I will teach you easier ways to calculate CIs in R soon.

Practical Matters

Reporting a CI

  • Always report point and interval estimates together

  • In text, use this format:

    • Statistic = Point Estimate, X% CI: [Lower Bound, Upper Bound]
    • \bar{x}=3.91, 95% CI [3.81, 4.00]

  • You can also put point and interval estimates in a table or a figure (though be sure to specify that they are X% CIs)

Example Table

Parameter Estimate 95% CI
Mean Eval 3.91 [3.81, 4.00]

Example Figure

Comparing CIs

  • A higher confidence level will result in a wider interval
    • To claim more confidence, the interval must “pay” with width
# Only the multiplier changes
qt(0.995, df = n - 1)   # 99%
## [1] 2.629732
qt(0.975, df = n - 1)   # 95%
## [1] 1.985802
qt(0.950, df = n - 1)   # 90%
## [1] 1.661404

# The 99% interval, then
xbar - (qt(0.995, df = n - 1) * se)
## [1] 3.781061
xbar + (qt(0.995, df = n - 1) * se)
## [1] 4.031067

Interpreting CIs

It is common to misinterpret what confidence intervals mean

Incorrect Interpretation

“There is a 95% probability that the population parameter value lies within the 95% CI.”

The above is incorrect because it does not align with the frequentist view of probability

Bayesian interval estimates (credible intervals) actually can be interpreted this way

Interpreting CIs

The correct frequentist interpretation is quite a mouthful

Correct Interpretation

“If we repeated the experiment over and over again (with different samples of the same size) and computed the 95% CI in each sample, then 95% of those intervals would contain the population parameter value.”

Alternatives that are reasonable shortcuts

Acceptable Interpretations

“We can be 95% confident that the 95% CI contains the population mean.”

“The 95% CI contains highly reasonable estimates of the population mean.”

The first alternative isn’t my favorite as it implies a view of probability as belief (which isn’t very frequentist)

Another example

Let’s estimate the mean sleep satisfaction of all grad students, using a sample of 16

Calculate the pieces

sleep <- read_csv("sleep.csv")
x <- sleep$satisfaction
n <- length(x)
se <- sd(x) / sqrt(n)
t <- qt(0.975, df = n - 1)

Estimate mean and interval

mean(x)
## [1] 68.75
mean(x) - (t * se)
## [1] 57.77683
mean(x) + (t * se)
## [1] 79.72317

“We estimate that the mean sleep satisfaction among all grad students is 68.75, 95% CI: [57.78, 79.72]. Thus, our best guess for this value is 68.75 but, given sampling error, any number from 57.78 to 79.72 would also be a highly reasonable guess.”