CRM Goals

  • Understand what research is and how it is done

    • Survey major types of research in criminology

    • How to ask good questions

    • How to answer questions effectively

  • Recognize unique challenges in criminological research
  • Learn to critically evaluate research
  • Teach you to design an empirical research project

What I cover

Mainly methodology:

  • Theory and process of answering research questions

Secondarily methods:

  • Procedures used to generate and interpret data

Split design:

  • First half emphasizes quantitative approaches
  • Second half emphasizes qualitative approaches

​Materials: clanfear.github.io/ioc_crm/

Criminological Research

 

Joining a conversation

Criminology is a conversation; to contribute, you need to know:

  • What the conversation is about
  • What the current state is
  • What its premises are

This requires understanding of theory and methodology and how they relate to each other

This understanding allows us to…

  • Ask good research questions
  • Develop good research designs to answer them
  • Evaluate others’ research

Good Research Questions

What makes a research question good?

  • It is empirically answerable1

    • It is in some sense factual
    • It is identifiable in principle

  • It improves our understanding of the world (i.e., it informs theory)

    • An answer would change how we think or behave
    • An answer would solve a significant problem

 

These are subjective: You must make an argument!

Sources of Questions

  • Theory

    • We have expectations—are they correct?
      • I think these expectations are wrong!
    • Are there competing expectations from different theories?
      • Adjudicate between them!

  • Ignorance

    • We have no expectations!
    • True scientific ignorance is very rare1
      • Usually: “is \(x\) like \(y\)?” (back to theory!)
      • Almost never: “nothing is like \(x\)”
    • Must make convincing argument no extant theory applies

 

In either case, you’re probably going to need to read a lot.

Types of Questions

  • Description - What does this look like?

    • How do crime rates differ across urban neighbourhoods?
    • How does exposure to violence vary over the life course?

  • Exploration - What is going on here?

    • Do violent neighbourhoods have different interpersonal networks?
    • Are children exposed to violence more likely to carry guns?

 

  • Explanation - Why is this happening?

    • Do strong interpersonal networks reduce neighbourhood violence?
    • Does childhood victimization increase gun carrying?

  • Evaluation - Does this work?

    • Does creating a neighbourhood organization reduce violence?
    • Does this child victims’ program reduce gun carrying?

 

Each of these requires a different research design!

Research Design

Research design is strategy for answering research questions:

  • What data are needed
  • How to obtain those data
  • How to analyze those data
  • How to interpret results

Good research design is that which can answer your (good) research question

There are many different types of empirical research design

The type you choose is determined by your question

Descriptive Research

  • Accurate and systematic description of something

  • Can take many forms, e.g.:

    • Incidences and prevalences

      • How often do Americans witness gun violence? . . .

    • Processes

      • How are police stop records generated? . . .

    • Experiences

      • How do incarcerated women experience reintegration? . . .


Description is the foundation of science: We must know what before we can explain why

What does exposure to gun violence look like over the life course?

Exploratory Research

  • Building insight into a phenomenon with limited data or expectations

    • No theory testing or causal claims
    • Is there something interesting going on here? When does this thing occur?

  • Often similar to descriptive research, but searching for:

    • Meanings: How is offending related to decision-making processes?
    • Potential explanations: Are officers convicted of misconduct different than others?
    • Predictions: What are risk factors for witnessing gun violence?

  • Often used to develop explanatory questions

    • Which of these risk factors are causes?
    • Do these meanings cause behaviour or are they a result of it?

How are online gang conflicts related to offline gang violence?

Exploratory ≠ Atheoretical

Reminder: There is no atheoretical research

Theory is never completely isolated from problems of empirical research, any more than empirical research is free from theoretical assumptions (Layder 1994:vi)
Theory is inextricably involved in the process of data-gathering and data-interpretation (Bottoms 2007:75)

Research assumes:

  • What is worth studying
  • What questions to ask
  • What counts as evidence
  • How to measure
  • How to analyze

Explanatory Research

  • What causes \(y\)? What does \(x\) cause?

    • Why did US gun violence increase in 2020?
    • Does perceived risk of apprehension reduce offending?

  • Deductive

    • Start with a theory
    • Collect data to test it
    • e.g., hypothesis testing

  • Inductive

    • Start collecting data
    • Develop theory from data
    • e.g., grounded theory

Theory testing and development is explanatory research!

CRM focuses on explanatory research designs

Explanatory Process

Does residential change reduce recidivism?

Evaluation Research

  • Is this intervention or policy effective?

    • Goals of intervention are normative: Some outcome is desired
    • Effectiveness may still be objective

  • Even if you can answer this question, you might not know why

    • Data may not be collected on what turns out to be important later

    • E.g. Minneapolis Domestic Violence Experiment

      • Arrest reduced domestic violence—but not in replications
      • Research design doesn’t reveal why

  • Explanatory research often done after or during evaluation research

    • Interviews and observations (e.g., mechanism examinations)
    • Secondary data analysis (e.g., heterogeneity tests)

Do body-worn cameras improve police stop outcomes?

Linking Together


Aside: Correlation ≠ Causation

We often hear correlation doesn’t imply causation, which is true…


How many people has Nicolas Cage drowned?

Can he be stopped?

Aside: Correlation ≠ Causation

We know correlation doesn’t imply causation

More problematic: Causation can occur without correlation!

  • Correlations tell us nothing about causes without theory
  • Be careful extrapolating from exploratory results
  • Beware research conflating association and causation

We’ll dive into this deeply in a couple weeks

Research Approaches

Qualitative and quantitative

A Qualitative Example

In this unique and original book, Peter St. Jean examines why some blocks in urban areas experience more crime than others. Based on a number of sources—most importantly, in-depth interviews with drug dealers and routine robbers about their strategies for selecting a location or victim…

  • Mario Small

How do offenders interpret disorder and act based on that?

A Quantitative Example

This important dataset on neighborhoods within each of ninety-one of the largest cities of the United States allows Peterson and Krivo to craft a structural race theory of neighborhood crime based on racial inequality, residential segregation, and spatial inequality.

  • Ross Matsueda

How does urban racial inequality relate to crime in the United States?

Qualitative

Contextual, categorical, interpretive, inductive

Data

  • Interviews
  • Documents
  • Observation (Overt/Covert)
  • Participation (Overt/Covert)
  • Non-numeric

Strengths

  • Access hidden populations
  • Capture context and meaning
  • Can observe mechanisms
  • Easier to communicate

Challenges

  • Research time requirements
    • Scales poorly
  • Subjectivity at forefront
  • Risk and access
  • Replication / Generalizability

Quantitative

Abstract, numeric, hypothetical, deductive

Data

  • Surveys
  • Secondary data
    • Police & Government Data
    • Commercial Data
  • Field measurements

Strengths

  • Replication / Generalizability
  • Testing and validating
  • Measuring effects
  • Scales well

Challenges

  • Up-front time requirements
  • Homogenization
  • Data demands
  • Hidden subjectivity
  • Harder to communicate

Feedback Loop


Mixed Methods Research

Research in which the investigator collects and analyses data, integrates the findings, and draws inferences using both qualitative and quantitative approaches or methods in a single study or program of inquiry (Tashakkori and Creswell 2007:4)

  • Intellectual and practical synthesis

  • Common in applied research

  • Qualitative data analysis helps:

    • Make sense of observations or statistical data
    • Guide the formulation of hypotheses to be tested
    • Provide evidence for mechanisms

A Methodological Toolbox

Some specialization is necessary but overspecialization is limiting, both intellectually and professionally. It is important to be able to:

  • Interpret findings of any kind
  • Collaborate with others with different specialties
  • Critique research using any approach

And another good reason to have a strong – and more importantly, eclectic – methodological skill‐set is so that you will not have to define (and therefore confine) yourself as either a quantitative or qualitative criminologist (although some people enthusiastically embrace such labels). You can instead call yourself a criminologist and be safe in the knowledge that you have command of whatever methodological “tool” you will need to answer whatever criminological question you have decided to ask. (Pratt 2015)

What would you do?

Causality

Figure 1: Theoretical model. CE is collective efficacy, O is criminal opportunity, C is crime, A is short-term lettings, U is omitted confounders

Causality

Scientific theories are causal explanations about how the world works

  • \(y\) occurs because \(x\)

Thus all explanatory research is causal research

Quantitative half of CRM emphasizes causality:

  • Does \(x\) make \(y\) happen?

Causality isn’t necessarily about numbers, nor factors that are physical, behavioural, or even directly observable: Meanings can be cause actions; experiences can cause meanings

We must know causes to make interventions, e.g., controlling crime

But what is a cause anyway?

Causes and Mechanisms

A cause is something that can initiate a mechanism to produce an effect

  • \(x\) causes \(y\) via the mechanism of \(z\)

A mechanism is a process linking input to output under specific conditions

  • How a cause works
  • Mechanisms specified by theory but often not observable

Causes and Mechanisms

 

the hallmark of modern science is the search for lawful mechanisms behind the observed facts, rather than the mindless accumulation of data and the mindless search for statistical correlations among them. (Bunge 2004)1

Causes and Mechanisms

A cause is something that can initiate a mechanism to produce an effect

  • \(x\) causes \(y\) via the mechanism of \(z\)

A mechanism is a process linking input to output under specific conditions

  • How a cause works
  • Mechanisms specified by theory but often not observable

Example: “..collective efficacy may also reduce crime by empowering residents to remove or prevent the development of sources of criminal opportunities…” (Lanfear 2022)

Boxes and Arrows

Graphs (“boxes and arrows”) are commonly used to illustrate theories:



  • Boxes are things we specify and measure
    • In quantitative research, we call these variables
  • Arrows represent one or more mechanisms
  • \(x\) → \(y\) means “X causes Y”
    • Absence of an arrow means no causal relationship

Graphs are powerful tools for explanatory research—we’ll see this soon!

An Example


Broken Windows Theory (Wilson & Kelling 1982):

They get complicated


… but imagine explaining this only in words!

Testing Theories Deductively

  1. Start with research question
  2. Generate hypothesis
    • A theory-based falsifiable prediction
  3. Test hypothesis using data
    • Quantitative statistical hypothesis testing (week 3!)
    • Qualitative hypothesis testing (e.g., event studies and case-comparisons)
  4. Re-evaluate theory
    • Falsified hypotheses are evidence against theory1
    • Repeated falsification of hypotheses should falsify theories2

Testing Theories Deductively

Testing Theories Deductively

  1. Start with research question
  2. Generate hypothesis
    • A theory-based falsifiable prediction
  3. Test hypothesis using data
    • Quantitative statistical hypothesis testing (week 3!)
    • Qualitative hypothesis testing (e.g., event studies and case-comparisons)
  4. Re-evaluate theory
    • Falsified hypotheses are evidence against theory1
    • Repeated falsification of hypotheses should falsify theories2

Let’s see some examples

Broken Windows

An Observation: Serious crime is more common in neighbourhoods with disorder, such as litter, panhandlers, and loitering youth

The Broken Windows Thesis:

  • Offenders interpret disorder as a sign of low social control (mechanism)

    • If no one stops littering or pan-handling, no one will stop robbery or drug dealing

  • Offenders are thus more likely to commit crime

A Proposition: At the neighbourhood level, disorder causes crime

What questions might we ask and how might we test them?

An Experiment

Keizer et al. (2008): Does seeing disorder make individuals more likely to commit a crime?

  • Leave money-filled envelope dangling from postbox

  • Record whether passersby mail or steal the envelope

  • Add litter or graffiti and repeat

  • Compare rates of theft

Theft was twice as common with either litter or graffiti.1

Interpreted as evidence for the broken windows thesis

Observational

Sampson & Raudenbush (1999): What if low social control causes both disorder and crime?

  • Measure disorder and social control in 80 Chicago neighborhoods

  • Test if disorder predicts crime when social control is held constant1

No effect of disorder on crime once accounting for social control

Interpreted as evidence against the broken windows thesis

Interviews

Broken windows thesis says offenders interpret disorder as a sign of low social control

St. Jean (2007):

  • How do robbers and drug dealers interpret disorder?

  • Why do they choose particular locations for crime?

Offenders don’t care about disorder, they care about opportunity, of which social control is only one factor

Interpreted as evidence against the broken windows thesis—and for a more complicated social control theory

Comparing Designs

What do these have in common?

  • Testing same theory
  • Following same approach
    • Derive hypotheses from theory
    • Test with data
    • Re-evaluate theory

How do they differ?

  • Different (but related) questions
  • Studying different outcomes
    • i.e., behavior vs. rates vs. meanings
  • Using different methods
  • Different assumptions
    • e.g., litter mimics disorder; meanings determine behavior

All three provide useful answers!

Wrap-Up

  • Theory is inseparable from methodology
  • Choice of methods should be driven by the question
  • Theory testing is causal inference

Readings for next session:

  • Bernard, T.J. (1990) ‘Twenty years of testing theories: what have we learned and why?’, Journal of Research in Crime and Delinquency, 27(4): 325–347. http://dx.doi.org/10.1177/0022427890027004002

  • Wikstrom, P-O H. (2017) ‘Character, circumstances, and the causes of crime: Towards an analytical criminology’ In A. Liebling, S. Maruna and L. McAra (eds) Oxford Handbook of Criminology, 6th edition. Oxford University Press. Pg 501-521