Alphoria AIAI systems are built by people, trained on data collected by people, and deployed to make decisions that affect people. At every step, the assumptions, gaps and prejudices that exist in human society have the potential to enter the system and become baked into its outputs. This is what researchers mean when they talk about bias in AI — and it is one of the most important concepts for children (and adults) to understand about the technology.
Start with a story: the lollipop jar
Here is an analogy that works well with children of most ages. Imagine a jar full of lollipops. Most of them are red, a few are blue, and one is green. You close your eyes and pull out five. You get four red ones and one blue. You have not seen a green one, so your brain starts to think: "lollipops are mostly red, and sometimes blue. Green might not exist."
Now imagine you are an AI, and the lollipops are people in photographs. If the photos you were trained on mostly showed one type of person, your "brain" would start to assume that type is the default. When you meet people who are not well represented in your training data, you would make worse decisions about them — not because you were told to, but because you never saw enough of them to learn properly.
That is bias from unrepresentative training data, and it is one of the most common sources of AI bias in real-world systems.
Real examples (explained for children)
Face recognition that struggles with darker skin tones. Several widely-used face recognition systems were trained mostly on photographs of lighter-skinned people. When tested on darker skin tones, their accuracy was significantly worse — sometimes to the point of wrongly identifying the wrong person. This has caused real harm in cases where police used such systems in criminal investigations.
Hiring tools that preferred men. An AI tool developed by a large technology company to help sort job applications learned from the company's own historical hiring decisions — which, like many technology companies, had historically hired more men than women. The AI learned to rank male applicants more highly. The tool was abandoned when this was discovered.
Medical AI that missed symptoms in darker skin. Some AI tools designed to detect medical conditions from skin photographs were trained predominantly on images of lighter skin. They performed worse when used on patients with darker skin, potentially leading to missed diagnoses.
In each case, the AI was not "trying" to be unfair. It was doing exactly what machine learning systems do: finding patterns in the data it was trained on. The data reflected inequalities in the world — who got hired, who got photographed, who was included in medical studies — and the AI faithfully reproduced those inequalities in its outputs.
The three main sources of AI bias
Training data bias. The data used to train an AI does not represent the world equally. Some groups of people, places, languages or situations appear far more than others. The AI learns what it sees — and is worse at what it rarely encountered.
Measurement bias. The way we measure things shapes what we count and what we miss. If an AI is trained to predict "success" at a job based on historical performance reviews, and those reviews were themselves biased (given less favourably to women, or to people from certain backgrounds), then the AI learns to replicate those biased assessments.
Feedback loops. AI systems that affect the real world and learn from that real world can amplify initial biases. A predictive policing algorithm that sends more police to one neighbourhood will gather more crime data from that neighbourhood (because police are there), which trains the algorithm to send even more police there — not because crime is actually higher, but because the system created a self-reinforcing cycle.
What children can do with this knowledge
Understanding bias in AI is not about becoming cynical about technology. It is about becoming a thoughtful user — and eventually, perhaps, a thoughtful builder. Three questions worth teaching:
"Who decided what this AI should do?" Every AI system was designed to achieve a goal that someone chose. That choice reflects the values and priorities of the people who built it. Asking who those people are, and whether their perspective matches your own, is the start of critical thinking about AI.
"Who might be left out?" Whenever an AI makes a decision about people, ask: who might this system work less well for? Are there groups of people who are less represented in its training data, or whose needs were not considered in its design?
"What is the cost of getting it wrong?" The stakes matter. A recommendation algorithm that suggests the wrong film is a low-stakes error. An AI system that wrongly flags someone as a credit risk, or incorrectly identifies them in a criminal database, can cause serious harm. Higher stakes demand higher scrutiny.
The bigger point
AI bias is not inevitable, and it is not unsolvable. Researchers and engineers work specifically on making AI systems fairer — testing them on diverse populations, using more representative training data, and building tools to detect bias before systems are deployed. But that work depends on people who understand the problem and demand it be taken seriously.
Children who grow up knowing that AI systems can be unfair, and understanding why, are better positioned to demand better — as users, as citizens, and eventually as the people who build the systems that come next.