class: center, top, title-slide # Measuring Crime and Policing ## SOC371 ### Chuck Lanfear ### Jan 20, 2021<br>Updated: Jan 19, 2021 --- # Overview 1. Research Approaches 2. Measuring Crime * Police Data * Victimization * Self-Report 3. Measurement and Levels of Explanation * Individual Characteristics * Groups and Places 4. Measuring Relationships * Types of Effects * Measuring Bias in Policing --- # Take Methods Courses Today is a brief overview of Sociological methods. These topics are too much to cover in one term let alone a day. -- To get a foundation in these topics, I recommend you... * Take a course in *research design* + SOC300: Foundations of Social Inquiry + SOC403: Applied Research * Take a course in *research methods* + SOC/STAT/CS&SS221: Statistical Concepts and Methods + SOC321: Data Science and Statistics ??? Research design is about how to think about and approach a problem--essentially, how you write a question that can be answered. Methods are about how you answer that question in a convincing way. If you ask a poor question, answering it well doesn't benefit anyone. Most of the advanced methods you see in modern papers aren't seen in any undergraduate classes; sometimes not even in grad classes. --- class: inverse # Research Approaches ### Qualitative and quantitative .pull-left[ .image-75[  ] ] .pull-right[ .image-75[  ] ] ??? It is necessary to clarify the main types of approaches to research. Important to know difference. --- # A Qualitative Example .pull-left[ .image-75[  ] ] .pull-right[ > In this unique and original book, Peter St. Jean examines why some blocks in urban areas experience more crime than others. Based on a number of sources--most importantly, in-depth interviews with drug dealers and routine robbers about their strategies for selecting a location or victim... - Mario Small ] .centernote[*Goal: Understand how offenders **interpret** world and act based on that.*] ??? One of my favorite crim books is primarily qualitative. St. Jean spent enormous amount of time talking to residents, police, and most importantly active drug dealers, robbers, and people regularly involved in violence. Goal was to understand how offenders choose where--and how--to conduct their criminal activities. Fundamentally about meaning and decision-making. The data here are narratives of offenders. --- # A Quantitative Example .pull-left[ .image-75[  ] ] .pull-right[ > This important dataset on neighborhoods within each of ninety-one of the largest cities of the United States allows Peterson and Krivo to craft a structural race theory of neighborhood crime based on racial inequality, residential segregation, and spatial inequality. - Ross Matsueda ] .centernote[*Goal: Examine how urban racial inequality leads relates to crime in the United States.*] ??? We'll be reading this later in the term. Book is the result of a massive undertaking in numeric data collection--getting crime rates for every census tract in nearly 100 cities, analyzing relationship to structural conditions. This is about examining a numerical relationship: How does segregation relate to crime throughout the country? The data here are numbers capturing inequality and counting crime. --- # Qualitative *Contextual, categorical, interpretive, inductive* Data * Interviews * Observation (Overt/Covert) * Participation (Overt/Covert) * *Non-numeric* ??? Qual work is, by definition, non-numerical and non-statistical. It focuses on narrative and interpretation. -- .pull-left[ Strengths * Access hidden populations * Capture context and meaning * Observe mechanisms * Easier to Communicate ] ??? Qual work often inductive: looks at specific cases and extrapolates to general theory; the specific to the general Qualitative work can get at lived experiences--how people make sense of world Qualitative work often only way to get offender perspectives Populations you want to reach can be risky or difficult to access But can ask and answer questions from perspective of agents involved Can capture whole processes including motivations and feelings of actors involved -- .pull-right[ Challenges * Research time requirements + Scales poorly * Subjectivity at forefront * Risk and access * Replication / Generalizability ] ??? Qual work typically huge time commitment Must address subjectivity--different researchers will interpret situations differently; have to make a strong argument that others would interpret it the same way... and that you'd see the same thing in other similar situations. Possible to get swept into ethical issues: Alice Goffman's *On The Run* + Most charitable interpretation is being uncritical of informant claims, getting swept up in their lives + Less charitable (but possibly true) is substantial fabrication and unethical behavior --- # Quantitative *Abstract, numeric, hypothetical, deductive* Data * Surveys * Secondary data + Police Data + Government Data ??? Quant work is be definition numeric--it makes numeric comparisons and evaluations Quant work often deductive: Begins with general rules, tests hypotheses and applies to cases; the general to the specific Might also use novel things like sensors or social media data -- .pull-left[ Strengths * Replication / Generalizability * Testing and validating * Measuring effects * Scales well ] ??? Quant research can typically be reproduced computationally, replicated statistically, and more easily generalized to new contexts Can get precise, actionable numeric estimates for policy Can use gross amounts of data that are impossible to handle qualitatively -- .pull-right[ Challenges * Up-front time requirements * Homogenizes population and context * Data demands * Hidden subjectivity * Harder to communicate ] ??? Learning to do quant research well is an enormous undertaking--and even the best make major mistakes Can easily miss important heterogeneity; can also miss important mechanisms that would be obvious from qualitative research Data needs to be available--this means small or hidden pops go unstudied Different researchers will make different choices in data cleaning and modeling --- # Feedback Loop .image-short[  ] ??? This is an idealized model--quant work can uncover some mechanisms, qual work often reveals anomalies. There's also some great qualitative work that tests propositions! Can be dismissed by pure quant people--but ignore that dogma In general though, it is inadvisable for any area to focus too heavily on either--creates blind spots Also wastes effort as people focus on the wrong mechanisms (quant) or overgeneralize rare cases (qual) Both are filled with bad takes --- class: inverse # Measuring Crime --- # Police Data Uniform Crime Report * Agency-period units * Hierarchy rule * High participation ??? Police data is administrative data--a form of secondary data collected by agencies for their own uses. It isn't generated for research purposes, so there is little concern for things researchers care about like sample representation Talked about UCR already--agencies report index crimes as counts for the whole agency in a time period -- National Incident Based Reporting System * Incident units * No hierarchy * Lower participation -- Agency Data * Unique to department --- # The Dark Figure of Crime .pull-left[ * Not all crimes are discovered + Victimless crimes + Missing person homicides * Not all crimes are reported + Varies by place + Varies by individuals + Varies by crime * Not all reported are recorded by police + Varies by agency + Varies by crime + Political incentives + Differential treatment ] .pull-right[ .image-short[  ] ] ??? The Wire: Police are often under pressure to reduce the crime rate; easiest way to do that is avoid charging people; juking stats If police believe DA won't pursue, they won't either This means better enforcement makes crime go up! Differential treatment may produce bias not seen in actual commission of crimes --- # An Example *Observed Crime Rate* `\(=\)` *Actual Rate* `\(*\)` *Probability of Reporting* <br> -- > According to the U.C.R., the incidence of rape nearly doubled from 1973 to 1990. The N.C.V.S., by contrast, shows that it declined by around forty per cent during the same period. Researchers... found that the upward trend in the U.C.R. data correlated with upticks in the number of female police officers, and with the advent of rape crisis centers and reformed investigative styles. It could be, in short, that a modernized approach to the policing of rape drastically increased the frequency with which it was reported while reducing its incidence. [Matthew Hutson. 2020. "The Trouble with Crime Statistics." *The New Yorker*](https://www.newyorker.com/culture/annals-of-inquiry/the-trouble-with-crime-statistics) ??? In this case many things mattered for determining apparent rape rates Police practices, police identities, victim characteristics, and supporting services Changing understandings of consent, fear of victim blaming, cultural support likely to have effects too Good lay person article --- # Victimization Data .pull-left[ Advantages * Captures unreported crime * Can get at exposure ] -- .pull-right[ Challenges * Reluctance to answer * Hidden populations * Relived trauma * Sampling: Expense and coverage * Doesn't capture crime without victim ] ??? If you survey a person, you get their victimization; hard to track over time, or many people Crime without victim includes consensuals but also homicide Missing and murdered indigenous women and girls -- Examples: National Crime Victimization Survey Seattle Neighborhoods and Crime Survey Durfee. 2011. "'I’m Not a Victim, She's an Abuser'": Masculinity, Victimization, and Protection Orders." --- # Self-Reported Offense Data .pull-left[ Advantages * Captures unreported crime * Captures crime without victim * Can get at motivation ] -- .pull-right[ Challenges * Reluctance to answer * Hidden, even dangerous, populations * Easier to get youth than adults * Many crimes very rare * Sampling: Expense and coverage ] ??? Random sampling means basically impossible to reach enough for national, hard even in city Targeted samples have limitations Separation between incarcerated and free samples -- Examples: National Longitudinal Survey of Youth Add Health St. Jean. 2007. *Pockets of Crime: Broken Windows, Collective Efficacy, and the Criminal Point of View* --- # Ideal Data *Ideal criminological data would cover every potential crime and what occurred as a result* .pull-left[ That is, for a given opportunity, was the crime... * Attempted? * Successful? * Discovered? * Reported to Police? ] -- .pull-right[ And was the offender... * Seen or Found? * Informally Sanctioned? * Arrested? * Incarcerated? ] -- .centernote[*These are all important but nearly never all observed, especially not for a large number of incidents.*] ??? In the real world we only observe very particular combinations of these things. What is left out can be very important for how we interpret findings. --- class: inverse # Measurement and Levels of Explanation --- # Types of Measures ‍Observables: Objective, externally measurable variables * Individual height, occupation, stated opinions * Neighborhood population, geographical area, crime rate<sup>1</sup> .footnote[[1] Hard to measure doesn't mean unobservable!] *Observables can be directly seen and/or experienced* -- Unobservables (Constructs): Subjective, internal, or only indirectly measurable * Individual self-control, wellness, social class, education * Neighborhood social capital, disadvantage, collective efficacy *Unobservables do not exist in the real world: they are conceptual* -- In Criminology, we're *very often* interested in unobservables. ??? Unobservables of individuals are usually not even internally verifiable or measurable. Unobservables can mean something different to everyone. These are ideal types. Some theoretically observable things must be treated as unobservable. Sometimes unobservables are treated as observable. --- # Measurement Concepts ‍Conceptualization: *What is X?* * Define what a measure constitutes * Define relations to other concepts * Goal: Minimize ambiguity ??? Important thing here is being clear about what is meant--what you want to measure What does social class mean? Should capture place in social hierarchy of society; should be related to things like how you speak, what media you consume, social and political beliefs. -- ‍Operationalization: *How do I measure X?* * Define observable(s)--**indicators**--that are related to the concept * If single indicator, it is a **proxy** for the concept + e.g. *Years of School* is a proxy for *Education* * If multiple indicators, they can be combined into *composite* measure<sup>1</sup> When reading articles, pay attention to how things are measured! .footnote[[1] Composites include things like indices, scales, factors, and principle components. Turning multiple measures into one composite is sometimes called *dimension reduction*.] ??? Conceptualization and operationalization are tricky and can lead to lots of arguments When you read articles, try to follow the logic. If there's a disconnect between concept and measures, it might signal a problem. If someone wants to know the effect of social class in crime but only uses income, they're looking at the association of income. If income + education, they're looking at some average of those two. Are those social class? Consider if there are high class people with low educations and incomes and low class people with high educations and high income. --- ## Individual Example ### Self-Control ‍Concept: The ability to delay gratification, tolerate frustration, and carefully consider before acting. -- ‍Measures: *Would you strongly agree, somewhat agree, or disagree that you...* * "Get upset when you have to wait for something?" * "Act without stopping to think?" * "Like to do daring things?" * "Are impatient--want to have things right away?" * "Are careful about what you do?" -- ‍Assumption: *Shared variation in the measures represents underlying self-control.* --- ## Neighborhood Example ### Expectations for Child-Centered Informal Control ‍Concept: The shared neighborhood norms and expectations for intervening against child misbehavior. ??? This is a complex one that we'll see a bunch in social disorg--mainly collective efficacy Idea is capturing neighborhood capacity to control child behavior in public spaces Isn't about what respondent would do--is about what people around would do -- ‍Measures: *How likely<sup>1</sup> is it that people in your neighborhood would stop it if...* * "a group of neighborhood children were skipping school and hanging out on a street corner." * "some children were spray-painting graffiti on a local building." * "children were fighting out in the street." * "a child was showing disrespect to an adult." .footnote[[1] (1) Very Likely, (2) Likely, (3) Unlikely, (4) Very Unlikely] -- ‍Assumption: *Shared variation in the measures represents underlying expectations for social control.* --- class: inverse # Measuring Relationships --- # Types of Relationships * ‍**Association**: *X and Y tend to "go together"* + Ex: Ice cream sales and violent crime rise at same time + *May* imply common causes, e.g. temperature + *Does not* imply banning ice cream will reduce crime or vice versa + Purely observational--makes no assumptions -- * ‍**Effect**: *X causes Y* + Ex: Clearing a vacant lot reduces nearby violence + Implies direction and cause: + Clearing more lots will reduce violence there + Reducing violence will not clear lots + Requires strong assumptions to identify -- To measure an *effect*, you need to determine the difference in an outcome (Y) due solely to the cause of interest (X). --- # Establishing Causation ‍Problem: We only observe *one outcome* per unit--if a person takes a pill, we don't see what happens when they don't take it. ??? Fundamental problem of causal inference -- ‍Solution: Randomized Treatment (An experiment!) * On average, randomly selected groups differ only by treatment. * Average outcome difference between groups is treatment effect. ??? Randomization is incredibly powerful, overcoming almost any inferential obstacle where possible -- .image-threequarterwidth[  ] ??? People may differ in any number of ways, but as long as those differences don't impact the likelihood of receiving treatment, you can get an accurate measure. --- # Establishing Causation ‍*Hard Problem*: Most things we're interested in can't be randomly assigned. * We can't assign rough childhoods or neighborhood incomes. * Other things related to our outcome (crime) are also related to cause of interest. .image-threequarterwidth[  ] ??? Many things aren't just practically unassignable--many are impossible, like race. There's a vigorous debate in literature over whether something which cannot be manipulated (in principle) can be a cause Can't manipulate race... but can manipulate *perception* of race -- ‍*Hard Solution*: Measure everything else related and adjust for it. ??? Hard to measure or even *know* everything related This is basically the big barrier in observational, quantitative work In qualitative work it can be easier to establish cause--but much harder to generalize --- class: inverse # Measuring Bias in Policing ### An example from Knox, Lowe, & Mummolo. 2020. "Administrative Records Mask Racially Biased Policing" --- # Use of Force ‍Question: Do police use more force against black civilians? ??? Note the effect here is really perceptions of race by police, not race itself. -- Imagine you have sample of police encounters identical except for race. *But* suppose bias leads police to: 1. Stop white civilians only for serious crimes 2. Stop black civilians with or without crime -- Then, discard data on anyone police observed but *did not stop*. + You are now comparing use of force against white people committing serious crime to black ones committing no crime + If use of force were the same, *we'd have a serious problem* -- *This is what police data actually show!* * We only see the stopped people. * There is no comparison group. --- # What We Want .image-threequarters[  ] ‍Question: How does race impact use of force? --- # The Problem .image-threequarters[  ] Problems * Race impacts likelihood of stop * Race also impacts use of force * Can't control for stops *because we never see unstopped people* --- # It Gets Worse .image-threequarters[  ] More problems * Racial composition varies by neighborhood * Police deployments and strategy vary by neighborhood * Suspicion predicts both stops and use of force--and can't be observed ??? Some novel papers have gotten at these before, but it is incredibly challenging. One example is comparing drivers stopped when sun is up to when sun is down--when police can't see race. --- # Consequences * Detecting bias in the decision to stop is difficult + Hit rate tests ??? Hit rate tests estimate differences between likelihood of, for example, a frisk resulting in a found gun by race of suspect If guns found less often on black suspects, indicates bias Not perfect because need to adjust for fact that black folks live in different places and police strategies differ One important thing is using more objective outcome: arrests might be biased, finding a weapon can't be biased unless they planting. -- * If any stop bias exists, it is difficult to measure bias in... + Arrests + Use of force + Frisks + Shootings ??? This is an issue many if not most researchers working in this area don't fully grasp Vast majority of work in area does not account properly for this -- * Raw numbers can easily show *opposite patterns* from underlying reality. * Studies showing no bias--or anti-white bias--get *lots* of media, social media, and political traction. ??? Fryer paper making the rounds back last summer showed this, but was just wrong. --- # Summary * Numbers *don't* speak for themselves ??? Administrative data are particularly problematic They hide bias and can also be manipulated -- * Methods should match the question ??? If you want to measure an effect or estimate the extent of something, it is probably quantitative If you want to observe a mechanism, understand the meaning of things or motivations of people, it is probably quantitative Most questions benefit from both--a common good practice is using quant methods but then doing qual work to examine cases that don't fit the model -- * Methodological Things to Pay Attention to + Make sure concepts and operationalizations match up + Consider omitted variables + Consider missing data ??? Measures are important--bad measure invalidates everything after If you leave important things out, it can invalidate your results too -- * Interesting questions are hard to answer ??? The most important questions in sociology, criminology, and public policy are either difficult or impossible to answer with an experiment. Sometimes they're difficult or impossible to answer with observational data either. --- class: inverse # Questions --- # For Next Time * Marshall, Chris E. 2002. “Deterrence Theory.” Pp. 512-515 in *Encyclopedia of Crime and Punishment*. Edited by D. Levinson. Beverly Hills: Sage * Levitt, Steven D. 2002. “Deterrence.” Pp. 435-450 in *Crime: Public Policies for Crime Control*. Edited by J.Q. Wilson and J. Petersilia. Oakland, CA: ICS press ### Things to pay attention to: * Connection to classical school * Theoretical assumptions of deterrence * Challenges for measuring deterrence * Policy implications of deterrence