AP Computer Science Principles · Unit 2 lecture · Part 8 of 11 · 8 min

2.3 Extracting information from dataFrom numbers to knowledge

Mr. Thompkins
We live in a world drowning in data: every tap, purchase and step gets recorded. Data on its own isn't useful, though. The skill is getting true information out of it, and not fooling yourself along the way.

Information is the collection of facts and patterns extracted from data. Data provide opportunities for identifying trends, making connections and addressing problems. A bike-share company with a year of trip records can see which stations run out of bikes at 8 a.m., when demand peaks, and where to add new stations.

Often a single source doesn't contain the data needed to draw a conclusion, so it may be necessary to combine data from a variety of sources. Trip data plus weather data can show how rain changes riding. Trip data plus repair records can show which bikes are busy and overdue for service.

Bike IDTrips this week (app data)Last repair (shop data)Combined insight
B-104413 weeks agoBusy and recently fixed
B-221388 months agoBusy and overdue: repair first
B-31741 week agoRarely used: move it to a busier station
Combining two data sets that share a bike ID produces information neither one holds alone.

Metadata

Metadata are data about data. For a photo, the data is the pixels, and the metadata is when and where it was taken, the camera, and the file size. Metadata are used for finding, organizing and managing information: your phone can show every photo from a trip because each one carries a location and a date. And changes to the metadata don't change the primary data: renaming a song or fixing its genre tag doesn't change a single sound sample.

Part of a song fileData or metadata?
The sound samples you hearData: the song itself
Title and artistMetadata
Length (3:42)Metadata
Date you added itMetadata
Genre tagMetadata
For a song, the sound is the data. Everything that describes it is metadata.
?Wait, is a file's name data or metadata?
Metadata. The name describes the file; it isn't the content. You can rename a file and its contents stay exactly the same, which is the classic sign of metadata.

Correlation is not causation

Digitally processed data may show a correlation between variables: when one goes up, the other tends to go up (or down) too. A correlation found in data does not necessarily mean that a causal relationship exists, that one thing makes the other happen. Additional research is needed to understand the exact nature of the relationship.

0306090120150180051015umbrellas sold in a dayminor car crashesthey rise together
Eight days of city data. Days with more umbrellas sold also have more minor crashes. Strongly correlated, but do umbrellas cause crashes?

Of course not. Rain causes both: people buy umbrellas, and wet roads cause crashes. A hidden third factor like this is the most common reason a correlation isn't a cause. Sometimes the cause runs the other way, and sometimes a correlation is a coincidence. Big data sets make it easy to find patterns; they don't make it easy to explain them.

RainUmbrella salesCar crashescorrelatednot cause and effectcausescauseslook for a third factor before claiming a cause
A third factor, rain, drives both measurements. They move together without one causing the other.
?Wait, what would it take to show a cause?
Usually a controlled experiment: change only one thing, keep everything else the same, and see whether the other thing changes. In Unit 2 you only need to know that data alone can show correlation, and that claiming causation needs more investigation.
Pause and think
A study finds that students who eat breakfast have higher grades. A headline says "Breakfast raises grades." What's wrong with the headline?
I thought about it. Show me
The data shows a correlation, not a cause. A third factor could explain both, such as families with more time or money in the morning, or students who sleep more. More research, like an experiment, would be needed to claim breakfast causes higher grades.
Data, metadata, and what you can conclude
A city collects bus records: each bus ID, each stop, and arrival time, plus a log of when each record was uploaded and by which device. Analysts notice that buses with newer engines arrive on time more often. (a) Name one piece of data and one piece of metadata. (b) What can the city conclude about newer engines, and what can't it?
  1. (a) Data: the arrival time of bus 12 at Elm Street. Metadata: the upload time and device for that record.
    Why?
    Metadata describe the records; they aren't the bus information itself.
  2. (b) It can conclude that newer engines and on-time arrival are correlated in this data.
  3. It can't conclude the engines cause the on-time arrivals. A third factor might: perhaps newer buses are assigned to shorter, less crowded routes.
    Why?
    Correlation does not imply causation. Name a possible third factor to show you understand why.
  4. To learn more, the city could combine sources (route length, traffic data) or try newer engines on the slow routes and compare.

Answer: (a) Data: an arrival time. Metadata: when and by which device a record was uploaded. (b) Newer engines and punctuality are correlated, but that doesn't prove the engines cause it; route differences could explain it.

Stuck? Get a hint
  1. Is the item about the buses, or about the records of the buses?
  2. Could something else make newer buses arrive on time?
  3. Correlation is what the data shows. Causation needs more investigation.
Check yourself
A music app lets users edit the genre label of a song. Which statement is true?
  1. AChanging the genre changes the audio data in the file.
  2. BThe genre is metadata, so changing it does not change the song's audio.
  3. CThe genre is the primary data, so the song must be re-recorded.
  4. DMetadata cannot be changed once a file is created.
Show the answer

B. The genre describes the song, so it is metadata, and changes to metadata don't change the primary data, the sound samples.

  • Information is the facts and patterns extracted from data. Combining sources often reveals more.
  • Metadata are data about data. Used to find, organize and manage data; changing them doesn't change the data.
  • A correlation in data does not necessarily mean causation. More research is needed.
  • Look for a hidden third factor before you trust a cause-and-effect claim.