AP Computer Science A · Unit 4 lecture · Part 1 of 14 · 8 min

4.1-4.2 Data: ethics, bias and using data setsBefore the code: where data comes from

By the end: Store, traverse and change data in arrays, ArrayLists and 2D arrays, read data from text files, run and trace the standard algorithms (linear and binary search, selection, insertion and merge sort), trace recursion, and think about where data comes from and who it leaves out.
Mr. Thompkins
Welcome to Unit 4, the biggest unit on the exam: 30 to 40 percent of your score lives here, and two of the four FRQs. Everything in it is about collections of data. So before we store a single number, let's ask where the numbers came from.

A data set is a collection of related data. Most data sets look like a table: each row is a record, one thing that was measured (one rider, one sale, one day), and each column is a field, one attribute of it (the stop, the wait, the rating). Programs read data sets to answer questions and find patterns: What's the average wait? Which stop gets the worst rating? Is it getting better over time?

riderstopminutes waitedrating (1-5)
R01Oak Ave64
R02Main St112
R03Oak Ave3003
R04Pine Rd4
R05Main St92
A small data set from a bus survey. Rider R03 waited 300 minutes? Rider R04 left the rating blank.

Look at the table before you compute anything. A 300-minute wait is almost certainly a typo, maybe 30 or 3. R04 never gave a rating. Real data has errors and holes like these, and a program has to decide what to do with them: skip the record, flag it, or ask someone. What it must not do is quietly treat a blank as a real value.

?Wait, how do you compute an average again?
Add up the values and divide by how many values you added. For ratings 4, 2, 3 and 2, the sum is 11 and there are 4 of them, so the average is 11 / 4 = 2.75. In Java, cast to double before dividing, or int division throws the .75 away.
Pause and think
Suppose a program counts R04's blank rating as 0. What happens to the average rating, and why is that a problem?
I thought about it. Show me
It drops from 11 / 4 = 2.75 to 11 / 5 = 2.2. The riders didn't rate the buses worse; the program invented a 0. A missing value is not a zero, so the right move is to leave R04 out of the rating average and say that one rating was missing.

Privacy: data is about people

Most interesting data sets describe people, and that brings responsibilities. Personal data, such as names, addresses, phone numbers, health records or a phone's location history, can reveal a lot about someone, especially when several data sets are combined. Once collected, data can be shared, sold, stolen in a breach, or used for something nobody agreed to. Good practice: tell people what you collect and why, collect only what the question needs, keep it secure, and remove anything that identifies a person when you don't need it.

?Wait, what counts as personally identifiable information?
Personally identifiable information (PII) is anything that can identify a specific person: a name, an address, an email, an ID number, a face, a precise location. Even harmless-looking pieces, like a birth date plus a zip code, can identify someone when they're combined.

Bias: who is in the data

A program's answers are only as good as its data. Data is biased when it doesn't fairly represent the whole group the question is about. Suppose the bus survey runs only in a smartphone app, in the morning. Riders without phones and night-shift riders are never asked, so their routes look empty, and a city that cuts "unused" routes could hurt exactly the people the survey missed.

everyone who rides the bussurveyed in the appnever asked:riders without a phonenight-shift ridersriders who skip the appa sample that leaves groups out gives a biased answer
The survey reached only part of the riders. Whatever the data says, it says about them.
Where bias sneaks inExampleWhat helps
Who gets askedOnly app users take the surveyCollect from every group the answer is about
Who answersOnly angry riders bother to replyFollow up; compare who answered with who rides
How it is asked"How bad was the delay?"Neutral wording and choices
What is missingNo data from 10 p.m. to 6 a.m.Say what the data cannot tell you
Old patternsPast routes served some neighborhoods poorlyCheck results for unfair patterns
Common ways bias gets into a data set, and what helps.

Notice that none of these problems is a bug in the code. A perfectly correct average of biased data is still a biased answer. That's why programmers ask where data came from, who collected it, how, and who is missing, before they trust what their program says.

Spot the problems
A city wants to decide which bus routes to cut. It asks riders to fill out a survey in a smartphone app between 7 and 9 a.m. The app also records each rider's name, home address and GPS trail for the day. Identify one source of bias and one privacy risk, and suggest a fix for each.
  1. Bias: the survey only reaches people with smartphones who ride in the morning. Night riders and riders without phones aren't represented, so their routes will look less used than they are.
    Why?
    Ask "who is the question about?" (all riders) and "who is in the data?" (morning app users). The gap is the bias.
  2. Fix for the bias: also use data that covers everyone, such as automatic passenger counts from every bus all day, or paper surveys on night buses.
  3. Privacy risk: a name plus a home address plus a GPS trail shows where a person lives and goes all day. If it leaks or is shared, it could be used to track them.
  4. Fix for the privacy risk: collect only what the question needs, such as the stop and time of each ride, with no name or address, and don't keep the GPS trail.
    Why?
    The question is about routes, not about people. Data you never collect can't leak.

Answer: Bias: only morning smartphone users are surveyed, so other riders' routes look unused; add all-day passenger counts or paper surveys. Privacy: names, addresses and GPS trails can track people; collect only stops and times, without identities.

Stuck? Get a hint
  1. Who rides the bus, and who answered the survey?
  2. Which riders can't use a smartphone app between 7 and 9 a.m.?
  3. For privacy, ask which collected data the route question doesn't actually need.
Check yourself
A school wants to know how crowded the cafeteria gets during each lunch period. Which plan answers the question while best protecting student privacy?
  1. ACount how many people enter during each period, without recording who they are
  2. BScan every student's ID card with the time they enter
  3. CPhotograph every table every five minutes
  4. DTrack every student's phone location during lunch
Show the answer

A. The question is how many, not who, so a plain count answers it and collects no personal data. The second choice is the top trap: it also answers the question, but it records exactly who ate when, which the question doesn't need.

  • A data set is a collection of related data, often rows (records) and columns (fields). Programs use it to answer questions and find patterns.
  • Check data before trusting it: errors and missing values must be handled, never counted as real values.
  • Personal data carries privacy risks. Collect only what you need, protect it, and remove what identifies people.
  • Biased or incomplete data gives unfair or wrong results even when the code is perfect.