Topic 5.3 Notes – Computing Bias
1. What Computing Bias Is
Bias is a tendency that leads to unfair or prejudiced outcomes. In computing, bias happens when a computing innovation (program, algorithm, AI model, app, etc.) systematically favors or disadvantages certain groups.
This is not about random errors. Bias shows up as patterns.
For example:
- A hiring system that consistently ranks applicants from one demographic group lower.
- A loan approval algorithm that denies loans to certain neighborhoods more often.
- A medical AI that performs worse on patients from underrepresented backgrounds.
Key ideas you need to be able to explain:
- Bias is often unintentional.
- It can create unequal or discriminatory outcomes.
- It may reinforce existing inequalities.
- It can exist even if the programmer had no harmful intent.
On tests, it’s not enough to define bias. You’ll need to explain how a specific computing innovation reflects or produces bias.
2. Where Bias Comes From in Computing
Bias can be embedded at every stage of development. Think about the full machine learning lifecycle shown below.
Machine learning lifecycle
From defining the problem to monitoring the system after deployment, each stage is a point where human decisions are made.
Let’s break that down.
a. Bias in the Data
Many systems use historical data to make predictions. If the data reflects past discrimination, the system can learn and repeat it.
Common data issues:
- Unrepresentative data
Some groups are underrepresented. If a speech recognition system is trained mostly on one accent, it may struggle with others. - Historical bias
If past decisions were unfair, the data captures that unfairness. A predictive policing tool trained on biased arrest data may send more patrols to the same communities. - Incomplete data
Missing context can distort results. If a dataset lacks key socioeconomic factors, predictions may oversimplify complex realities.
Core principle:
Biased data in → biased results out.
On multiple-choice questions, the AP often describes a dataset and asks why the outcomes are skewed. Look for clues about representation or historical inequality.
b. Bias in the Algorithm or Design
Even with balanced data, design choices matter.
Bias can come from:
- Variables chosen
What inputs are included? What’s ignored? - Objective function
What is the system trying to optimize? Profit? Speed? Accuracy? Optimizing one goal can harm another group. - Thresholds and cutoffs
A small change in a decision boundary can affect one group more than others. - Assumptions made during development
Developers decide what “normal” looks like. Those assumptions shape the output.
Algorithms reflect the priorities and values of their creators. They are not neutral just because they are mathematical.
c. Bias from Human Decisions During Development
Humans influence every step shown in the lifecycle:
- Defining the problem
- Choosing what data to collect
- Deciding what success looks like
- Selecting who participates in testing
- Monitoring (or failing to monitor) after release
Bias can be embedded at all levels of software development, not just in the code itself.
That phrase “all levels” is important. The AP likes asking where bias occurs, and the correct answer is usually broader than students expect.
3. How Computing Bias Leads to Unintended Consequences
Computing innovations are built with a purpose, but bias can create unintended consequences.
Examples of consequences:
- Disproportionate harm to certain groups
- Reduced access to jobs, housing, or credit
- Reinforcement of stereotypes
- Loss of trust in technology
- Discrimination at a larger scale than a single human decision
Because algorithms can operate at massive scale, biased systems can spread inequality quickly.
Another tricky idea: biased outputs may seem objective because “the computer decided.” That perception can make the bias harder to question.
When you’re asked to explain an unintended consequence, structure your thinking like this:
Source of bias → How it affects outputs → Who is impacted → What broader effect it creates.
4. How Developers Can Reduce Bias
Reducing bias requires both technical and human action.
a. Use Diverse and Representative Data
- Include data from all relevant groups.
- Actively seek out underrepresented populations.
- Update datasets regularly to avoid outdated patterns.
b. Review and Test for Bias
- Compare results across demographic groups.
- Look for unequal error rates.
- Conduct audits before and after deployment.
Do not assume neutrality. Test for fairness intentionally.
c. Use Fairness Measures
Developers can apply fairness metrics, such as checking whether approval rates are similar across groups. Making fairness measurable makes it easier to detect disparities.
d. Address Human Bias in Development
- Increase diversity on development teams.
- Include ethical review processes.
- Question assumptions during design.
- Ask who could be harmed by this system.
Programmers have a responsibility to take action to reduce bias. The learning objective explicitly expects you to recognize that.
