1. What Does Statistics Do?
Statistics provides methods for organizing data, describing its main features, and making estimates or decisions. Descriptive statistics summarizes observed data, while inferential statistics uses a sample to learn about a larger population.
| Area | Purpose | Examples |
|---|---|---|
| Descriptive statistics | Summarize observed data | Mean, median, mode, variance, standard deviation |
| Inferential statistics | Estimate or test population characteristics | Confidence intervals, p-values, hypothesis tests, regression |
2. Mean, Median, and Mode
| Measure | Meaning | Best used for |
|---|---|---|
| Mean | Sum divided by the number of observations | Numeric data without influential outliers |
| Median | Middle value after sorting | Skewed data such as income or prices |
| Mode | Most frequent value | The most common value or category |
For the waiting times 2, 3, 3, 4, 18:
Mean = (2 + 3 + 3 + 4 + 18) ÷ 5 = 6
The median is 3 and the mode is 3. The unusually long 18-minute wait pulls the mean upward, while the median and mode remain unchanged.
3. Variance, Standard Deviation, and Sigma
Measures of center do not describe spread. The datasets 4, 5, 6 and 0, 5, 10
both have a mean of 5, but the second is much more dispersed.
| Term | Meaning |
|---|---|
| Deviation | Difference between an observation and the mean |
| Variance | Average squared deviation |
| Standard deviation | Square root of variance, expressed in the original unit |
| σ (sigma) | A symbol commonly used for population standard deviation |
For 2, 4, 6, the mean is 4 and the squared deviations are 4, 0, 4.
Population variance σ² = (4 + 0 + 4) ÷ 3 ≈ 2.67
Population standard deviation σ = √(8/3) ≈ 1.63
Dividing by n describes a complete population or the observed dataset itself.
Estimating population variance from a sample commonly uses n−1.
4. P-values and Hypothesis Tests
A p-value is the probability of obtaining a result at least as extreme as the observed result, assuming the null hypothesis is true. It is not the probability that the null hypothesis is true.
- State the null hypothesis.
- Choose a significance level before inspecting the result.
- Select a test that matches the design and data.
- Calculate the test statistic and p-value.
- Reject the null hypothesis when
p < α.
The common 0.05 threshold is a convention, not a law of nature. Statistical significance is also different from practical importance. Examine effect sizes, confidence intervals, sample size, assumptions, and plots together with the p-value.
5. Regression Analysis
Regression expresses the relationship between an explanatory variable x
and a response variable y. Simple linear regression uses:
Predicted value ŷ = a + bx
The intercept a is the prediction at x=0, and the slope b
is the average change in y for a one-unit increase in x.
A regression association does not by itself establish causation.
6. The Least-Squares Method
A residual is the difference eᵢ = yᵢ − ŷᵢ between an observed and predicted value.
Ordinary least squares chooses the intercept and slope that minimize:
Sum of squared residuals = Σ(yᵢ − ŷᵢ)²
Squaring prevents positive and negative errors from cancelling, but also makes the fit sensitive to outliers. Inspect the slope, R², p-value, residuals, and uncertainty rather than judging a model from one number.
7. A Practical Order for Reading Data
- Check sample size, missing values, and input errors.
- Summarize the center with the mean, median, or mode.
- Measure spread with variance and standard deviation.
- Inspect distributions and outliers with appropriate plots.
- Choose a hypothesis test or regression model that matches the question.
- Report uncertainty, effect sizes, and diagnostics—not only p-values.
8. Key Takeaways
- The mean, median, and mode describe the center in different ways.
- Variance, standard deviation, and sigma describe spread.
- A p-value is defined under the assumption that the null hypothesis is true.
- Regression represents relationships between variables.
- Least squares minimizes the sum of squared residuals.
- Always inspect the data and the model assumptions.
References: NIST/SEMATECH Statistical Glossary, NIST/SEMATECH: Critical Values and p Values