Reassessing ‘The Effect of the Seattle Police-Free CHOP Zone on Crime’

A counterfactual critique

Charles C. Lanfear

University of Cambridge

The Capitol Hill Occupied Protest

They went into Seattle, they took over a big percentage of the city of Seattle (Donald Trump, 2024)

CHOP timeline

 

 

The CHOP lasted 24 days: June 8th to July 1st

Piza & Connealy (2022)

 

Their question

 

Did SPD’s withdrawal increase crime?

 

The geographically focused, time limited nature of CHOP provides an opportunity for a natural experiment to test the effect of the autonomous zone on crime. Given the context of CHOP, this study may further be considered a test of police abolition, the most radical proposal advanced in the police defunding movement. (Piza & Connealy 2022:36)

Piza & Connealy’s strategy

 

Microsynthetic control approach:

  1. Treat police withdrawal as quasi-experimental treatment
  2. Match 36 “treated” street segments1 to similar untreated “controls” among >20,000 candidate segments across city using:
    • Time-invariant place characteristics
    • Crime counts in 6-day periods covering prior year
  3. Compare total crime counts in treated and untreated groups in four 6-day periods—the 24 days of the CHOP2

Matching variables

Continuous:

  • 6-day period police-recorded crime counts (the only time-varying measure)
  • Count of commercial businesses
  • Count of consumer-facing facing establishments
  • Block group crime density
  • Length of the street segment

 

Binary:

  • Principal vs arterial street
  • If pre-CHOP total crime > 80th percentile
  • If police beat > 80th percentile of police calls
  • In block group with > mean disadvantage
  • In block group with > mean non-white residents, residents 15-29, vacant homes, and owned/rented homes

Their findings

Results indicate crime significantly increased in the CHOP zone, the encompassing two-block area, and the overall East precinct service area. (Piza & Connealy 2022:35)

The current study found comparatively stronger effects than the general literature on depolicing. The significant crime increase is particularly noteworthy given the short time frame of the CHOP occupation and retreat of police from the area theoretically making it more challenging for crimes to be reported by citizens and/or proactively discovered by officers. This suggests that police abolition, the most extreme form of the police defunding movement, may significantly compromise public safety. (Piza & Connealy 2022:35–36)

Disclosure

I was contracted by the City of Seattle expert legal report in a lawsuit by Seattle business owners against the City of Seattle for damages related to the police withdrawal from the East Precinct:

 

CC Lanfear. 2022. Expert Report: Hunters Capital, LLC v. City of Seattle (client). Case No. 20-cv-00983 TSZ, US District Court, Western District of Washington

 

This work is adapted from my expert report, with permission from the client.

Counterfactuals

Potential Outcomes


Ignorability


What is our counterfactual?

 

Causal inference hinges on having the right counterfactual.

 

What if we don’t know what \(Y_{A}^{0}\) we want?

What if the \(Y_{B}^{0}\) we have is not similar to \(Y_{A}^{0}\)?

 

This is Piza & Connealy’s problem

Revisiting Piza & Connealy

Implicit causal model

G chop CHOP Zone crime Crime chop->crime place Measured Place Characteristics place->chop place->crime protest Protest protest->chop protest->crime

Assumptions:

  • Primary: Protest is independent of CHOP Zone and/or Crime
  • Secondary: No unmeasured Place Characteristics are relevant

Problems

  • Protest definitely caused CHOP Zone and probably caused Crime
    • It is not conceivable the police pullout would have occurred absent the large-scale protest and repeat conflicts with police
    • The police pulled out because clashes with police were escalating
    • Protest caused massive increase in ambient population
  • Measured Place Characteristics don’t explain why Protest or CHOP Zone occurred
    • Places with similar characteristics–including crime–did not have large-scale protests
    • Capitol Hill has unique social-ecological characteristics associated with protest

Protest Capitol Hill

Typical “time-invariant” Capitol Hill

Capitol Hill is where protests happen

 

Seattle’s protest stood out from the national activity due to their extreme magnitude and duration, particularly1 in the context of the Capitol Hill neighborhood. (Piza & Connealy 2022:36)

Capitol Hill is an unusual area of prone to protests and conflicts with police

Crime spiked in Capitol Hill absent police withdrawal on July 25th, 2020 due to another protest.

What is their counterfactual?

Given these issues, what are Piza & Connealy estimating?

 

\(Y_A^1\) they observed

How many crimes occurred after police withdrew to de-escalate conflict with an existing large-scale protest against police

 

\(Y_B^0\) they estimated

How many crimes would have occurred if neither the protest nor the withdrawal occurred1

 

Critically, the difference between these does not tell us if withdrawing from the East Precinct increased crime

What counterfactual did they want?

Piza & Connealy’s (2022) counterfactual is only implied:1

A more prudent solution [than withdrawing] would have been to more effectively engage with protesters to quell the violent activity. (52)

Implied \(Y_B^0\): How many crimes would have occurred if police had remained but de-escalated the conflict

But recall SPD withdrew to de-escalate after exhausting options:

SPD abandoned the precinct in an attempt to quell the confrontations, resulting property damage, and injuries to both police officers and protestors. (36)

Police-reported crime in CHOP zone is in large part due to police activity:

  • Higher before withdrawal
  • Highest when SPD returned

Plausible counterfactuals

Even if they could estimate it, their implied counterfactual is implausible

But other realistic counterfactuals exist:

Alternative \(Y_B^0\): How many reported crimes would have occurred had…

  • SPD continued battling protestors?1
  • SPD suppressed the protest?
  • The national guard been deployed?2

Is it likely these counterfactuals would produce similar conclusions?

Takeaways

In general

  • Know and be clear about counterfactuals
  • Counterfactuals come from theory, so know your theory
  • Be critical consumers and reviewers

 

This paper

  • Estimate does not correspond to any useful estimand
  • Provides no information on depolicing effects
  • Other cities may offer valid counterfactuals (e.g., Portland, Minneapolis)

Feedback and Questions

Contact:

Charles C. Lanfear
Institute of Criminology
University of Cambridge
cl948@cam.ac.uk