Data: Practical Tips from the Field

Data: Practical Tips from the Field

With Ken Schwencke - @schwanksta Sisi Wei - @sisiwei Derek Willis - @derekwillis

The Story

The Cycle

• Obtaining the data
• Exploring the data
• Designing for the data
• Preparing the data
• Building the app

Obtaining the data

• Offline vs. Online
• Dumps vs. Feeds
• Data Entry: The Final Frontier

Exploring the data

• The Interview
• What's missing?
• Finding the Limits

Designing for the data

• Tell the story
• Keep it clear, not simple or complicated
• Watch for accuracy

Preparing the data

• Bringing in Other Sources
• Data as a Service

Building the app

By now, we've decided how much weight we're tossing behind the project.

For a big story, or an ongoing ("evergreen") project, we might choose to do a web app.

• Homicides in the District
• Homicide Watch DC
• Homicide Report

For a smaller story, we might go with a single-serving page

Even just a simple animation can be all you need:

Looping back

What if the addresses are all wrong?

Go back to the beginning with new assumptions and design from there.

How to manage data without driving everyone nuts

• Git
• PANDA
• SQLite

Lessons for Teaching

1. Data is just another source, and like any other source has strengths and weaknesses.

2. Data on the Web may not be the only data out there.

3. All data is dirty.

4. Use Excel as a gateway drug for data interviewing/exploration.

5. Ask simple questions first, then more complex ones.

6. Show students what is possible, and have them replicate or localize existing projects.

7. Have them work in groups. More sets of eyes is better.

8. Maps are great. You don’t always need or want one.

9. Avoid the Stockholm Syndrome - resist thinking that everyone knows the data like you do.

10. Data isn't static anymore.

Links, etc.

Links to Examples

Handout