Topic 3.5 Notes – Introduction to Experimental Design

1. How Experiments Establish Causation

An experiment deliberately imposes a treatment on individuals and measures a response. The key difference from an observational study is that treatments are imposed; when they are imposed by random assignment, the comparison is fair. Because the researcher controls who gets what treatment, experiments can justify cause-and-effect conclusions.

If treatments are imposed and randomly assigned, you can say the explanatory variable caused the change in the response.

Core Components of an Experiment

Let’s ground the vocabulary clearly.

• Experimental units

  The individuals or objects receiving treatments.

  If they’re people, they’re often called subjects or participants.

• Explanatory variable (factor)

  The variable the researcher manipulates.

  Its specific values are called levels.

• Treatments

  The specific conditions applied to units.

  A treatment can be:

  • One level of a factor (e.g., 50 mg dose)
  • A combination of levels from multiple factors
• Response variable

  The outcome measured after the treatment.

• Confounding variable

  A variable that:

  1. Is related to the explanatory variable
  2. Affects the response variable
  3. Makes it unclear whether the treatment actually caused the effect

  If confounding isn’t controlled, you can’t confidently attribute differences in response to the treatment.

On quizzes, you’ll often be asked to identify all five of these in context. Be specific and use the scenario’s language.

2. Principles of a Well-Designed Experiment

Strong experiments share four features. Miss one, and the design weakens.

Comparison

There must be at least two treatment groups so responses can be compared.

Often one is a control group, which may receive:

• No treatment, or
• A baseline or placebo treatment

Without comparison, you have nothing to evaluate.

Random Assignment

Each experimental unit has an equal chance of receiving any treatment.

This:

• Balances known and unknown confounding variables
• Justifies causal conclusions

Do not mix this up with random sampling.
Random sampling → generalizing to a population.
Random assignment → cause-and-effect.

Replication

More than one unit per treatment group.

Replication:

• Reduces chance variation
• Makes differences more believable

One person per treatment is not an experiment.

Control of Confounding Variables

You try to:

• Hold some variables constant
• Use blocking when helpful
• Randomize the rest

A phrase teachers love because it works:
Control what you can, block what you can’t, randomize the rest.

3. Control Groups, Placebos, and Blinding

Control Group

A control group receives no treatment of interest or receives a placebo. It provides a baseline for comparison.

Placebo and Placebo Effect

A placebo is an inactive treatment that looks like the real one.

The placebo effect happens when subjects improve simply because they believe they are receiving treatment.

Without a placebo, improvements might be psychological rather than caused by the treatment itself.

Blinding

• Single-blind: Either the subjects or the researchers interacting with them don’t know treatment assignments.
• Double-blind: Neither subjects nor interacting researchers know.

Blinding prevents bias in:

• Behavior
• Reporting
• Measuring responses

AP questions often test whether you recognize when blinding is missing.

4. Completely Randomized Design

This is the simplest design.

All experimental units are assigned to treatments entirely at random.

A completely randomized design looks like this:

Completely randomized design

How Random Assignment Happens

1. Label each experimental unit.
2. Use a random number generator, random digit table, or draw chips.
3. Assign treatments according to the random results.

Group sizes might not be perfectly equal. That’s fine.

Random assignment tends to balance confounding variables, so differences in response can be attributed to the treatment.

5. Randomized Block and Matched Pairs Designs

Sometimes there’s a known variable that strongly affects the response. Ignoring it adds extra variation.

Randomized Complete Block Design

You:

1. Divide units into blocks based on a blocking variable (e.g., grade level).
2. Randomly assign treatments within each block.

The diagram below contrasts a completely randomized design with a blocked design. On the left, everyone is randomly assigned straight to treatment or control. On the right, subjects are first separated into blocks, then randomly assigned within each block.

Completely randomized design vs. randomized block design

Blocking ensures units within each block are similar at the start.
This separates natural variability from treatment effects and increases precision.

Matched Pairs Design

A special case of blocking with block size 2.

Two common setups:

• Match similar subjects into pairs, randomly assign one treatment to each.
• Each subject receives both treatments in random order.

Matched pairs control individual differences extremely well.