4.1-4.2 Data: ethics, bias and using data setsBefore the code: where data comes from

A data set is a collection of related data. Most data sets look like a table: each row is a record, one thing that was measured (one rider, one sale, one day), and each column is a field, one attribute of it (the stop, the wait, the rating). Programs read data sets to answer questions and find patterns: What's the average wait? Which stop gets the worst rating? Is it getting better over time?
| rider | stop | minutes waited | rating (1-5) |
|---|---|---|---|
| R01 | Oak Ave | 6 | 4 |
| R02 | Main St | 11 | 2 |
| R03 | Oak Ave | 300 | 3 |
| R04 | Pine Rd | 4 | |
| R05 | Main St | 9 | 2 |
Look at the table before you compute anything. A 300-minute wait is almost certainly a typo, maybe 30 or 3. R04 never gave a rating. Real data has errors and holes like these, and a program has to decide what to do with them: skip the record, flag it, or ask someone. What it must not do is quietly treat a blank as a real value.
?Wait, how do you compute an average again?
I thought about it. Show me
Privacy: data is about people
Most interesting data sets describe people, and that brings responsibilities. Personal data, such as names, addresses, phone numbers, health records or a phone's location history, can reveal a lot about someone, especially when several data sets are combined. Once collected, data can be shared, sold, stolen in a breach, or used for something nobody agreed to. Good practice: tell people what you collect and why, collect only what the question needs, keep it secure, and remove anything that identifies a person when you don't need it.
?Wait, what counts as personally identifiable information?
Bias: who is in the data
A program's answers are only as good as its data. Data is biased when it doesn't fairly represent the whole group the question is about. Suppose the bus survey runs only in a smartphone app, in the morning. Riders without phones and night-shift riders are never asked, so their routes look empty, and a city that cuts "unused" routes could hurt exactly the people the survey missed.
| Where bias sneaks in | Example | What helps |
|---|---|---|
| Who gets asked | Only app users take the survey | Collect from every group the answer is about |
| Who answers | Only angry riders bother to reply | Follow up; compare who answered with who rides |
| How it is asked | "How bad was the delay?" | Neutral wording and choices |
| What is missing | No data from 10 p.m. to 6 a.m. | Say what the data cannot tell you |
| Old patterns | Past routes served some neighborhoods poorly | Check results for unfair patterns |
Notice that none of these problems is a bug in the code. A perfectly correct average of biased data is still a biased answer. That's why programmers ask where data came from, who collected it, how, and who is missing, before they trust what their program says.
- Bias: the survey only reaches people with smartphones who ride in the morning. Night riders and riders without phones aren't represented, so their routes will look less used than they are.
Why?
Ask "who is the question about?" (all riders) and "who is in the data?" (morning app users). The gap is the bias. - Fix for the bias: also use data that covers everyone, such as automatic passenger counts from every bus all day, or paper surveys on night buses.
- Privacy risk: a name plus a home address plus a GPS trail shows where a person lives and goes all day. If it leaks or is shared, it could be used to track them.
- Fix for the privacy risk: collect only what the question needs, such as the stop and time of each ride, with no name or address, and don't keep the GPS trail.
Why?
The question is about routes, not about people. Data you never collect can't leak.
Answer: Bias: only morning smartphone users are surveyed, so other riders' routes look unused; add all-day passenger counts or paper surveys. Privacy: names, addresses and GPS trails can track people; collect only stops and times, without identities.
Stuck? Get a hint
- Who rides the bus, and who answered the survey?
- Which riders can't use a smartphone app between 7 and 9 a.m.?
- For privacy, ask which collected data the route question doesn't actually need.
- ACount how many people enter during each period, without recording who they are
- BScan every student's ID card with the time they enter
- CPhotograph every table every five minutes
- DTrack every student's phone location during lunch
Show the answer
A. The question is how many, not who, so a plain count answers it and collects no personal data. The second choice is the top trap: it also answers the question, but it records exactly who ate when, which the question doesn't need.
- A data set is a collection of related data, often rows (records) and columns (fields). Programs use it to answer questions and find patterns.
- Check data before trusting it: errors and missing values must be handled, never counted as real values.
- Personal data carries privacy risks. Collect only what you need, protect it, and remove what identifies people.
- Biased or incomplete data gives unfair or wrong results even when the code is perfect.