What happens to the confidence interval as the value of the sample standard deviation increases?

Remember that there is variability associated with your outcomes and statistics.

When you calculate a statistic based on your sample data, how do you know if the statistic truly represents your population? Even if you've selected a random sample, your sample will not completely reflect your population. Each sample you take will give you a different result.

Let's Look at an Example:

Suppose that you want to compare the mean age for those with and without an IV in the prehospital setting. You review the ambulance runs for the past two weeks and calculate a mean age of 10.4 years for those with an IV and 8.5 years for those without an IV. The difference between the two means is 1.9 years. From this, you might conclude that those receiving an IV were older on average.

Suddenly, it's not clear that there's an important difference in age between these two groups. Now suppose you collect the same data over the next six weeks. This time the average age for those with an IV is 9.2 years and the average age for those without an IV is 8.9 years, for a difference of 0.3 years. Suddenly, it's not clear that there's an important difference in age between these two groups. Why did your different samples yield different results? Is one sample more correct than the other?

Remember that there is variability in your outcomes and statistics. The more individual variation you see in your outcome, the less confidence you have in your statistics. In addition, the smaller your sample size, the less comfortable you can be asserting that the statistics you calculate are representative of your population.

Providing a Range of Values

A confidence interval provides a range of values that will capture the true population value a certain percentage of the time. You determine the level of confidence, but it is generally set at 90%, 95%, or 99%. Confidence intervals use the variability of your data to assess the precision or accuracy of your estimated statistics. You can use confidence intervals to describe a single group or to compare two groups. We will not cover the statistical equations for a confidence interval here, but we will discuss several examples.

Example

• Average pulse rate = 101 bpm; Standard Deviation = 50; N = 200
• 95% Confidence Interval = (94, 108)
  We are 95% confident that the true pulse rate for our population is between 94 and 108.
  Margin of error = (108 – 94) / 2 = ± 7 bpm

The confidence interval in the above example could be described at 94 to 108 bpm (beats per minute) or 101 bpm ± 7 bpm. Here the number 7 is your margin of error. For confidence intervals around the mean, the margin of error is just half of your total confidence interval width.

Sample Size and Variability

The precision of your statistics depends on your sample size and variability. A larger sample size or lower variability will result in a tighter confidence interval with a smaller margin of error. A smaller sample size or a higher variability will result in a wider confidence interval with a larger margin of error. The level of confidence also affects the interval width. If you want a higher level of confidence, that interval will not be as tight. A tight interval at 95% or higher confidence is ideal.

Examples:

• Average Scene Time = 5.5. mins; Standard Deviation = 3 mins; N = 10 runs
• 95% Confidence Interval = (3.6, 7.4)
  Margin of Error = ±1.9 minutes
• Average Scene Time = 5.5 mins; Standard Deviation = 3 mins; N=1,000 runs
• 95% Confidence Interval = (5.4, 5.6)
  Margin of Error = ± 0.1 minutes
• Average Scene Time = 5.5 mins; Standard Deviation = 15 mins; N=1,000 runs
• 95% Confidence Interval = (4.6, 6.4)
  Margin of Error = ± 0.9 minutes

rev. 05-Aug-2019

Link 1

Link 1
(Description of link)

Let's say you have a sample mean, you may wish to know what confidence intervals you can place on that mean. Colloquially: "I want an interval that I can be P% sure contains the true mean". (On a technical point, note that the interval either contains the true mean or it does not: the meaning of the confidence level is subtly different from this colloquialism. More background information can be found on the NIST site).

The formula for the interval can be expressed as:

Where, Ys is the sample mean, s is the sample standard deviation, N is the sample size, /α/ is the desired significance level and t(α/2,N-1) is the upper critical value of the Students-t distribution with N-1 degrees of freedom.

Note

The quantity α   is the maximum acceptable risk of falsely rejecting the null-hypothesis. The smaller the value of α the greater the strength of the test. The confidence level of the test is defined as 1 - α, and often expressed as a percentage. So for example a significance level of 0.05, is equivalent to a 95% confidence level. Refer to "What are confidence intervals?" in NIST/SEMATECH e-Handbook of Statistical Methods. for more information.

From the formula, it should be clear that:

• The width of the confidence interval decreases as the sample size increases.
• The width increases as the standard deviation increases.
• The width increases as the confidence level increases (0.5 towards 0.99999 - stronger).
• The width increases as the significance level decreases (0.5 towards 0.00000...01 - stronger).

The following example code is taken from the example program students_t_single_sample.cpp.

We'll begin by defining a procedure to calculate intervals for various confidence levels; the procedure will print these out as a table:

We'll define a table of significance/risk levels for which we'll compute intervals:

Note that these are the complements of the confidence/probability levels: 0.5, 0.75, 0.9 .. 0.99999).

Next we'll declare the distribution object we'll need, note that the degrees of freedom parameter is the sample size less one:

Most of what follows in the program is pretty printing, so let's focus on the calculation of the interval. First we need the t-statistic, computed using the quantile function and our significance level. Note that since the significance levels are the complement of the probability, we have to wrap the arguments in a call to complement(...):

Note that alpha was divided by two, since we'll be calculating both the upper and lower bounds: had we been interested in a single sided interval then we would have omitted this step.

Now to complete the picture, we'll get the (one-sided) width of the interval from the t-statistic by multiplying by the standard deviation, and dividing by the square root of the sample size:

The two-sided interval is then the sample mean plus and minus this width.

And apart from some more pretty-printing that completes the procedure.

Let's take a look at some sample output, first using the Heat flow data from the NIST site. The data set was collected by Bob Zarr of NIST in January, 1990 from a heat flow meter calibration and stability analysis. The corresponding dataplot output for this test can be found in section 3.5.2 of the NIST/SEMATECH e-Handbook of Statistical Methods..

As you can see the large sample size (195) and small standard deviation (0.023) have combined to give very small intervals, indeed we can be very confident that the true mean is 9.2.

For comparison the next example data output is taken from P.K.Hou, O. W. Lau & M.C. Wong, Analyst (1983) vol. 108, p 64. and from Statistics for Analytical Chemistry, 3rd ed. (1994), pp 54-55 J. C. Miller and J. N. Miller, Ellis Horwood ISBN 0 13 0309907. The values result from the determination of mercury by cold-vapour atomic absorption.

This time the fact that there are only three measurements leads to much wider intervals, indeed such large intervals that it's hard to be very confident in the location of the mean.