Topic 1.13 Notes – Experimental Design

What an Experiment Is

In an experiment, the researcher assigns treatments to experimental units and measures a response variable. That assignment piece is what makes experiments different from observational studies.

• Observational study
  • Researchers observe what already happens. You can find association, but causation is shaky because of possible confounding.
• Experiment
  • Researchers impose conditions. With good design, this can support cause-and-effect.

The pieces you need to identify:

• Experimental units are the individuals or objects getting the treatment. For humans, people often say subjects or participants.
• The factor is the explanatory variable being imposed.
• Levels are the values/categories of that factor.
• Treatments are the actual imposed conditions.
  • With one factor, treatments are just the factor levels.
  • With multiple factors, treatments are the combinations of levels.
• The response variable is what gets measured after treatment.

One very testable detail is this. The experimental unit is the level where random assignment happens. If whole classrooms are assigned a teaching method, the classroom is the experimental unit, even if you measure student test scores.

Treatments also need to be specific enough to copy later. “Give more water” is too vague. “Give 100 mL daily” is a real treatment.

A simple experiment often looks like this. Subjects are randomly assigned to an experimental group or a control group, measured before treatment, and then measured again after.

Basic randomized pretest-posttest experiment

Principles of a Well-Designed Experiment

There are four things AP Stats wants you to check first.

• Comparison means at least two treatment groups.
  • You need something to compare against.
  • A control group gives a baseline. It might get no treatment, a placebo, the standard treatment, or another comparison condition.
• Random assignment uses chance to assign treatments.
  • Its job is to make groups similar on extraneous variables, which are outside variables that affect the response.
  • This reduces confounding, where an outside variable is tangled with treatment assignment.
  • Do not mix this up with random selection. Assignment is for causation, selection is for generalizing.
• Replication means more than one experimental unit per treatment.
  • Repeated measurements on one subject are not replication.
  • Replication helps average out natural variability and unusual units.
• Direct control means holding known outside variables constant when possible.
  • Same room, same time, same measuring method, same dose schedule, and so on.
  • Control fixes variables. Randomization balances variables.

Placebos and blinding

A placebo is an inactive treatment that looks like the real one. The placebo effect is the difference between the average placebo response and the average no-treatment response.

Blinding helps after assignment:

• Single-blind means participants or the researchers interacting with them do not know who got which treatment.
• Double-blind means neither group knows.

Students often blend these ideas together. Keep them separate. Randomization fights confounding before responses happen. Blinding fights bias after treatments are assigned.

The Main Experimental Designs

Completely randomized design

All experimental units are assigned to treatments completely at random. Equal group sizes are common, but not required. This works well when units are already fairly similar.

Completely randomized design

Randomized block design

You first group units into blocks based on a variable related to the response, then randomize treatments within each block.

The point is precision. If the block variable matters, comparing treatments inside each block removes some extra noise.

Huge AP trap: every treatment must appear in every block. If one block gets only one treatment, that is confounding, not blocking.

Matched pairs design

This is a special randomized block design with exactly two treatments.

Two forms show up:

1. Match two similar units and randomly assign one treatment to each.
2. Give both treatments to the same unit in random order.

This is great when close matching is possible or when one subject can receive both treatments. In repeated measures, watch for carryover effects.

How to Describe or Set Up the Design

A full design description usually includes:

1. the experimental units
2. the factor or factors
3. all treatments
4. the response variable
5. the comparison/control group
6. the random assignment method with actual mechanics
   Example: “Label subjects 01 to 40 and use a random number generator to assign 20 to treatment and 20 to control.”
7. replication by stating how many units get each treatment
8. any direct controls
9. any blocking or matching and why that variable matters
10. any placebo or blinding details

“Subjects were randomly divided” is too vague for a strong answer.

Choosing the Right Design and What Conclusions You Can Make

• Use completely randomized when units are fairly homogeneous and no strong block variable stands out.
• Use randomized block when a known outside variable strongly affects the response.
• Use matched pairs when there are exactly two treatments and pairing or repeated measures will cut variability.

Your conclusion depends on assignment versus selection: