AP Computer Science Principles · Unit 2 lecture · Part 10 of 11 · 9 min

2.4 Using programs with dataFilter, transform, combine, visualize

Mr. Thompkins
Nobody reads a million rows by hand. Programs do the reading for us, and they do it with a handful of moves you can learn today. Each move is a short loop, and you can trace every one of them.

Programs can be used to process data, which lets users discover information and create new knowledge. The course names a set of processes. Filtering keeps only the items that meet a condition. Transforming changes every element the same way, like doubling every value or converting units. Combining or comparing data produces a single result, like a total, an average or the maximum, or joins two data sets. Sorting puts data in order. Visualizing turns data into charts, graphs or maps. Clustering and classifying group or label similar items.

ProcessWhat it doesExample
FilterKeep only the items that meet a conditionOnly bike trips of 30 minutes or more
TransformChange every item the same wayConvert every temperature to Fahrenheit
Combine or compareJoin data sets, or compute one valueTotal minutes; the longest trip
SortPut the items in orderTrips from shortest to longest
VisualizeShow the data as a chart or mapA bar chart of trips per day
Cluster or classifyGroup similar items, or label themCommuter trips vs weekend rides
The common ways programs process data, each with a bike-share example.
?Wait, what does APPEND do?
APPEND(aList, value) adds value to the end of aList, making it one element longer. Starting with an empty list, [], and appending inside a loop is how programs build a new list of results.

Here's a filter, written in the exam's pseudocode. It goes through a list of bike-trip lengths in minutes and builds a new list that keeps only the trips of 30 minutes or more. FOR EACH visits every element in order, and the IF decides which ones to keep.

1trips ← [12, 45, 7, 30, 22, 61, 15]2longTrips ← []13FOR EACH m IN trips4{5  IF (m ≥ 30)26  {7    APPEND(longTrips, m)28  }9}10DISPLAY(longTrips)11DISPLAY(LENGTH(longTrips))
  1. line 2 An empty list to collect the results
  2. line 5-7 Filter: keep a trip only if it is 30 minutes or more
output › [45, 30, 61] 3
A filter: keep the long trips.
mm ≥ 30?longTrips after
12false[]
45true[45]
7false[45]
30true[45, 30]
22false[45, 30]
61true[45, 30, 61]
15false[45, 30, 61]
The trace, one row per trip. 30 is kept, because the condition is ≥.
?Wait, what does LENGTH do?
LENGTH(aList) returns the number of elements in the list. After the filter, longTrips holds three trips, so LENGTH(longTrips) is 3.

A transform looks almost the same, but there's no IF: every element goes through the same formula and lands in the new list. Here every Celsius temperature becomes Fahrenheit. A chart of the result is a visualization, and patterns are often easier to spot in a picture than in a list of numbers.

1celsius ← [10, 25, 30, 0, -5]2fahrenheit ← []3FOR EACH c IN celsius4{5  APPEND(fahrenheit, c * 9 / 5 + 32)16}7DISPLAY(fahrenheit)
  1. line 5 Transform: the same formula applied to every item
output › [50, 77, 86, 32, 23]
A transform: one formula, applied to every element.
°C10Mon25Tue30WedThu-5Fri°F50Mon77Tue86Wed32Thu23Fri
The same five days in both units. The transform changes the numbers, not the pattern.

Real data work is iterative and interactive. You filter, look at the result, notice something odd, clean it, filter differently, chart it, and ask a new question. Tools like spreadsheets help: they organize data in rows and columns and make sorting, filtering and charting quick. Search tools help you find specific information efficiently in large data sets. The course doesn't ask about any specific spreadsheet functions, only about what these processes do and what they can reveal.

Pause and think
Using the trips list, what would a program display if it added up only the trips shorter than 20 minutes?
I thought about it. Show me
The trips under 20 minutes are 12, 7 and 15, so the total is 12 + 7 + 15 = 34. That combines a filter (under 20) with a combine step (a running total).
Trace a data program
This code processes the trips list. Trace it and give the output.
1trips ← [12, 45, 7, 30, 22, 61, 15]2count ← 03total ← 04FOR EACH m IN trips5{6  IF (m ≥ 30)7  {8    count ← count + 19    total ← total + m10  }11}12DISPLAY(total / count)
Filter and combine in one loop.
  1. Set up a trace table with columns m, m ≥ 30?, count and total. Both start at 0.
  2. m = 12, false. m = 45, true: count 1, total 45. m = 7, false. m = 30, true: count 2, total 75.
    Why?
    30 ≥ 30 is true, so 30 counts. A > would have skipped it.
  3. m = 22, false. m = 61, true: count 3, total 136. m = 15, false.
  4. After the loop, total / count = 136 / 3 = 45.33...
    Why?
    The program filters (≥ 30) and combines (a total and a count) to get the average length of a long trip.

Answer: It displays about 45.33: the average of the long trips 45, 30 and 61 (136 ÷ 3).

Stuck? Get a hint
  1. Only the trips that are at least 30 change count and total.
  2. Those trips are 45, 30 and 61.
  3. Add them, then divide by how many there are.
Check yourself
What is displayed when this code runs?
prices ← [4, 10, 7, 2]
result ← []
FOR EACH p IN prices
{
  APPEND(result, p * 2)
}
DISPLAY(result)
  1. A[8, 20, 14, 4]
  2. B[4, 10, 7, 2]
  3. C[8, 20, 14, 4, 4, 10, 7, 2]
  4. D46
Show the answer

A. This is a transform: each price is doubled and appended to result, in order. 46 would be the sum of the doubled prices, which this code never computes.

Check yourself
A city wants to see whether bike trips drop on rainy days. Which process would help most?
  1. ACompressing the trip data with lossy compression
  2. BCombining the trip data with daily weather data and visualizing trips per day by weather
  3. CDeleting all trips from rainy days
  4. DSorting the trips by bike ID
Show the answer

B. The question needs two sources, trips and weather, combined, and a chart makes the pattern visible. Deleting rainy days would remove exactly the data needed.

Exam tip from Mr. T

When a question shows a data program, trace it with a table, just like in Unit 3. Then name what it does in one word: filter (has an IF), transform (changes every item), or combine (produces one total, count or maximum).

  • Programs process data to reveal information: filter, transform, combine or compare, sort, visualize, cluster and classify.
  • Filter: FOR EACH with an IF that decides what to APPEND. Transform: APPEND a changed copy of every item.
  • Combining data sources and visualizing data often reveal patterns that a single list hides.
  • Data work is iterative: filter, look, clean, chart, and ask a better question.