Readings (Part I)
~ 45 Minutes
Portland’s Precrime Experiment and the Limits of Algorithms
13-25 min. Protip: You can watch YouTube videos at more than 1X speed.†
Optional Media.
If you want to learn more about some of the topics discussed in the video above, and you have some free time, you might enjoy the following.
- Reports from the AI Now Institute. The folks at AI Now are doing some of the best scholarship on issues of AI and bias, and their annual reports are a must read.
- The New Jim Crow by Michelle Alexander. This book isn't about AI, but given the lengthy quote in the video, it seems worth including here.
- I name checked John Pfaff above and said there would be a link below. So... I'll recommend his book Locked In is an interesting contrast to The New Jim Crow. It finds a cause for massincarceration that is not rooted in drug laws, think prosecutors, but he's rather active on Twitter and I'd be remiss if I didn't suggest you follow him there for his frequent tweets about how we have to address how we treat violent crimes.
- Real-Time Crime Forecasting Challenge. Here is the challenge's offical page. For a lookt at the complete rules, including definitions of PAI and PEI, check out the archival copy of an earlier version.
- Predictive policing algorithms are racist. They need to be dismantled from the MIT Technology Review.
- Santa Cruz, Calif., Becomes First to Ban Predictive Policing
- Black Lives Matter May Be the Largest Movement in U.S. History
- S3, Episode 1: The Precrime Unit, Hi-Phi Nation (Podcast).
- LA IG Report on "Data-Drive" policing strategies. This is a good primary document.
- The Los Angeles Police Department Says It Is Dumping A Controversial Predictive Policing Tool
- Pasco’s sheriff uses data to guess who will commit crime. Then deputies ‘hunt down’ and harass them from the Tampa Bay Times.
Acknowledgments: The video above was adapted from a piece I wrote for Lawyerist back in 2017 when they were structured as a law blog. Unfortunately, it is no longer available on their site. The faux minority report image is minority-report-omg-02 by youflavio is licensed CC BY-SA 2.0. The Precision-Recall graphic by Walber can be found here.
Readings (Part II)
~ 1.5 Hours
Knowledge Base
Everyone comes to this adventure with a different background. So this section is designed to be a menu of sorts. If you already know a topic well, you can skip the relevant material. Just answer the questions below, and section(s) will disappear accordingly. That being said, if a section doesn't disappear, you should do it. Any time you save skipping a topic, however, should be spent working on your final project or reading ahead in either Weapons of Math Destruction or How Not to Be Wrong. FYI, we will be reading all of Weapons of Math Destruction and all but parts III and V of How Not to Be Wrong.
All of that being said, let's see if we can pare things down.
Could you construct a confusion matrix from raw data?
Could you articulate a rational for using F1 Score instead of accuracy when evaluating a prediction/classification algorithm?
You've gained roughly 10 minutes by dropping a video introduction to the Confusion Matrix, Accuracy, Precision, Recall, and F1 Scores. If you find yourself with questions, change this answer to unhide the introduction.
The Confusion Matrix et al.: Accuracy, Precision, Recall, and F1 Scores. Oh my!
Video: 5-10 Minutes. Protip: You can watch YouTube videos at more than 1X speed.
This wikipedia article is a good reference for all of you confusion matrix needs. It links to discussions of Accuracy, Precision, Recall, and F1 Scores.
Your Mission: A Simple Housing Issue Spotter (Part I)
Video: ~15-30 Minutes. Protip: You can watch YouTube videos at more than 1X speed.
Additional Work: 10 Minutes
Here is a link to Scratch. And if your interest was piqued when I mention reworking classic arcade games, you can check out some of my creations here. ;) FWIW, I'm suggesting you don't spend more than 10 minutes working on your bot's rules.
Your Mission: A Simple Housing Issue Spotter (Part II)
Video: ~18-36 Minutes. Protip: You can watch YouTube videos at more than 1X speed.
Additional Work: 10 Minutes
Here is the link to Machine Learning for Kids (we're all kids at heart ;). And here are the two files I mentioned:
Note: If you aren't an enrolled student, and you DM me, I'll provide you with login info for the ML4Kids exercises if I have any accounts available.
Your Final Project
Enrolled students will be presenting on their final project in four weeks. So if you haven't spoken with Colarusso about your final project idea(s), or lack there of, you should touch base ASAP! See The Final Project Rubric.
Self-Reflection and Logging Your Work
~20 min
As we do at the end of every level, we ask that you take a few minutes to reflect on how things are going. I've also included a set of reading questions to queue things up for our synchronous discussion. Your answers will be shared with me and it will let me know that I can look for any project work you may have posted. That being said, you've almost completed Level 4. Tell me how it's going by completing the form linked below.
Synchronous Meet Up, AKA our Class Time
1 hour and 30 minutes | September 21, 2020 @ 4pm Eastern
If you're an enrolled student, we'll be meeting at this link on Monday September 21st at 4pm via Zoom. If you don't have the password, and you are a registered student, DM me on Slack, and I can give you the password. If you're not an enrolled student, I'm afraid you can't join us.
We will use this time to: (1) evaluate our bots; and (2) discuss the readings.
† Time estimates are just that—estimates. The assumptions used to calculate reading time are as follows: 48 pages is assumed to take roughly an hour to read. When working with non paginated texts, it is assumed that a page is roughly equal to 250 words. Videos assume both 2X and 1X viewing. Estimates for coding are based on past experience. Each level should include about 6 hours and 40 min of work.