Grad Student Beginner’s Guide to Restricted-Use Data

I decided to write down some steps for graduate students looking to work with restricted-use data, but not at a university or department offering dedicated secure data space or licenses to tack on to (because institutional memory is important!). My advisor was super helpful in walking me through the process, but this outline quick guide may be helpful to others, especially since this isn’t a topic you learn in any courses in graduate school.
There are two main steps to obtaining restricted-use data, which are:
1.) IRB approval at your university
2.) Restricted-use data application approval from the data source
The first step is to read about the IRB (Institutional Review Board) process and complete a CITI training, so that you understand the process and issues that arise in research (https://about.citiprogram.org/en/series/human-subjects-research-hsr/). As a researcher, you will renew this every three years. I actually found the history and ethical issues around Human Subjects Research in the training to be very interesting.
After that, some tips and steps:
1. Identify the research office on campus. Ask “Which office deals with Research Compliance and IRB approval?” Folks in this office can guide you to the necessary steps in getting your research approved and get rolling with your restricted-use data license.
2. Is your research human subjects research? If so, what category? To find this out, you will first file a Human Subjects Determination Form, or similar, with the research office (or equivalent).
3. After determining the category of your human subjects research, then you will begin the IRB approval process. This involves writing a protocol for your institution’s IRB that demonstrates why you need restricted-use data and how you will adhere to security protocols.
3. For your IRB protocol, collect all application materials from the data source and note the steps needed to complete them.
a. Do you need a computer security plan?
b. Do you need notarized signatures? Is a notary available in your department or on campus?
c. Does some portion of the application need to be completed by the Research office of your university?
4. Draft all of the relevant portions of your protocol for your records, and to be reviewed by your advisor. This includes:
a. Computer security plan
b. Outline of project and methods
c. Description of why restricted-use data is necessary for your project. Be clear and succinct in describing your research question, method, supporting literature, and why your project is a contribution in which analyzing the restricted-use data in necessary. This description is important, since reviewers from other fields may be reading your protocol, and you want to clearly communicate your project, especially to those less familiar with your specific research.
5. You will then submit your protocol for approval by the IRB. Once this is complete and approved, you will then complete the data license application from whichever restricted-use data source you are using (for example, the Institute of Education Sciences, Department of Labor, etc.) At this stage, the research office can assist with next steps if needed.
6. General advice: Ask around! Did your advisor or mentor previous use restricted-use data? Or another professor in your department, or another department? Ask for help about how the application process has gone for others.
7. More advice: When in doubt, just keep pestering and asking for help. In my experience, the bottleneck was finding physical space (i.e. a secure office space). But, asking and reminding proved very useful is finding a small, secure office space for using my data.
8. Helpful clarification: Many restricted-use data licenses require the applicant to hold a PhD, so you will need your advisor or professor to assist in your application. They will then specify you are a data user on the license agreement.
As a PhD student at a department without dedicated resources for secure data, it can feel like more “prestigious” programs have a leg up when it comes to accessing certain data. They do, but that should not deter you from a good research idea, and your advisors and professors are there to support you in pursuing that, especially if you are willing to put in the work of going through the license application process. I also look at it as a very good learning experience for when I apply for restricted-use data or similar research methods as an Assistant Professor, where I will once again have to learn the institutional processes and procedures.
Next steps after getting your data: PRINT THE USER MANUAL (if that feels feasible), find tutorials on your data, and look for replication code from others who have published with your data.

More writing about writing

I feel intrigued by this idea of “deep work” and limiting distractions, and the cognitive cost of those distractions. From experience in PhD world, it is certainly true that interruptions and distractions, including those tasks like email, can reduce work quality. But, I have also always found small “shallow work” to be important to start with as it builds momentum in the work process, sometimes. The tricky part though is to build that momentum and actually move into “deep work”: https://www.nytimes.com/2019/01/13/smarter-living/how-to-actually-truly-focus-on-what-youre-doing.html#

I found this blog post about writing peer reviewed research to be very useful. The idea of building “deep work” writing into your schedule everyday has been really transformative for me in this dissertation process, and greatly reduced my anxiety around writing. I definitely went from binge writer to everyday writer in the past few years, or as this post puts it binge vs. incremental writing. Some days are just 30 minutes of “low hanging fruit” tasks like citations, writing a summary of a useful paper, etc, while other days are many hours of planning, writing, formatting, or doing metrics work. A research log is crucial- it is where I plan out my work, reflect on my work, log what I have completed, and look ahead. I also really like the advice of one paragraph per subject. It’s easy to get lost in academic writing, and spend several paragraphs just trying to prove you know one small particular thing, but being succinct usually means you actually understand what you are writing and can convey it simply: http://phdtalk.blogspot.com/2018/02/lesson-learned-writing-peer-reviewed.html?m=1

 

Resources I’m reading on doing, presenting, and publishing research

How to publish applied economics papers

Take away points

  • Expect rejection, and rejection is okay!
  • Also expect to wait….
  • Know your journal, its aim and scope, who publishes and what is published
  • Craft not just good analysis, but also a good story
  • Place your research within the literature, filling a gap or expanding on an outstanding question
  • Sell your question, and answer
  • Write with style and flow
  • Dial in the elevator pitch, and turn this into your abstract and introduction
  • Don’t bury the question
  • Make tables and graphs readable, and meaningful
  • Focus on the question and the contribution
  • and look at the following……

The introduction formula

  • A good boiler plate model
  • Hook, question, antecedent, value-added, road-map

The conclusion formula 

  • Summary, limitations, relevance to policy, future research

How to give an applied micro talk 

  • Clear question
  • Preview the findings
  • THEN, go into detail, but spare the boring details about processing data
  • Present a clear and explicit model
  • …”no pauses unless you really want to stress something”
  • Use tables strategically
  • Practice!

Updated Abstract: Carceral Schools and College Expectations

Revising my research and my abstract:

This study examines the impact of attending a carceral school- that is a school with metal detectors, locker checks, police or security staff, required identification badges, locked doors, and surveillance cameras- has on students’ expectations that they will attend college in the future. Using pooled data from the National Crime Victimization Survey: School Crime Supplement 2005, 2007, 2009, 2011, 2013, and 2015, I show that visible and intrusive security measures negatively impact students’ expectations of their future educational attainment. Using a probit model, I estimate with significance that metal detectors are associated with a -0.024 marginal effect on a student’s expectation of attend any college after high school and a -0.021 marginal effect for expecting to graduate from a four-year college, while controlling for achievement, race and ethnicity, gender, age, private or public school, area of residence, parental education level, and household income level. Further, I show that for non-white students, the effects tend to be larger, and that for Black males, the presence of security guards or police has a negative effect on expectations of attending any college in the future. I survey the literature on the school-to- prison pipeline, school security, and racial identity and schooling to theorize that four channels contribute to this negative effect: the crowding out of college preparatory resources, internalized negative feedback and stereotype threat, discipline and the achievement gap, and developing perceptions of injustice with regards to school security. This finding illustrates an important negative spillover effect of increasing school security in regards to student’s expectations of their future educational attainment.”

New Studies on Schools and Policing: At what cost?

My dissertation explores some of the negative spillover effects of the “school-to-prison pipeline” dynamic- a trend encompassing increasing use and severity of school discipline, increased interaction with the criminal justice system in schools, zero-tolerance policies, and public schools that look more like prisons than educational facilities. I’m interested in understanding how these measures impact students, both those students directly impacted by discipline but also those exposed to these “carceral” school environments. One specific area to look at is how police in and around schools impact student outcomes. Do cops in schools make students safer, or are there negative spillover effects of creating a more carceral school environment?

A few new publications begin to explore the relationship between policing, schools, and student outcome. Owens (2015) uses administrative data on school resource officers and federal hiring grants to show that the placement of law enforcement officers in schools gives officers more information on crime in schools and increases the likelihood of arrests for those crimes. The study also shows that these additional hires aid in drug chargers both on and of school campuses and increase crime reporting in neighborhoods. Owens’ study however does not interrogate what the impacts on student outcomes are associated with increased SRO hiring, and if anything, illuminates the effectiveness of school policing in creating “school-to-prison pipeline” dynamics. Weisburst (2019) uses data on federal grants for Community Oriented Policing Services in Texas, to show that the increasing placement of police officers in schools associated with these grants increases middle school discipline rates by 6 percent, with Black students experiencing the largest increases. These grants are also associated with a 2.5 percent decrease in high school graduation rates and a 4 percent decrease in college enrollment. Legewie and Fagan (2019) use innovative data on policing surges associated with Operation Impact in New York City to show how these surges impact educational outcomes for students. Using administrative data, they show that exposure to these surges reduces test scores for Black boys, and the effect increases with age.

These studies begin to shed light on some of the distinctly negative impacts that school policing may have for educational outcomes. Specifically, these dynamics negatively impact Black students (often boys) more so than other groups, contributing to racial inequality and stratification. 

Short Abstract for “Carceral Schools & College Expectations: Evidence from the National Crime Victimization Survey School Crime Supplement”

This study examines the impact of attending a “carceral school”- that is a school with metal detectors, locker checks, police or security staff, required identification badges, locked doors, and surveillance cameras- has on individual students’ expectations that they will attend college in the future. Using data from the National Crime Victimization Survey: School Crime Supplement 2005, 2007, 2009, 2011, 2013, and 2015, I show that visible and intrusive security measures negatively impact students’ expectations of their future educational attainment. Using  a probit model, I estimate that metal detectors and locker checks are associated respectively with a significant -0.026 and -0.017 marginal effect on a student’s expectation of graduating from a four-year college in the future, while controlling for achievement, race and ethnicity, gender, age, private or public school, household income level, and time. I then survey the literature on the school-to-prison pipeline, school security, and racial identity and schooling to theorize that three channels contribute to this negative effect: the crowding out of college preparatory resources, internalized negative feedback and stereotype threat, and developing perceptions of injustice with regards to school security. This finding illustrates an important negative spillover effect of increasing school security in regards to student’s expectations of educational attainment.

The School Security Paradox: What are the costs?

My dissertation research broadly focuses on the impacts of the so-called “school-to-prison pipeline” dynamic in United States public schools. To me, this issue is at the crux of some of the country’s most important social issues: gun violence, racism, under-resourcing of public schools, privatization of schools, the military-industrial complex, and mass incarceration. What I am calling the “school security paradox” emerges out of the intersection of two issues shaping our public school environments: gun violence and mass shooting events most often perpetrated by lone white males, and the unfair and disproportionate use of school discipline, school security, and the juvenile criminal justice system against minority- mainly Black- students.

Most of the literature on the emergence of the school-to-prison pipeline names the 1980s War on Drugs, the Columbine shooting, and the 1994 Gun Free Schools Act as main catalysts for changes in school policies and environments. These changes include increasing reliance on school security such as metal detectors, zero-tolerance discipline policies, and the outsourcing of discipline to the juvenile criminal justice system. These reasons though beg the question: if preventing another Columbine was the motivation for creating “safer” schools via discipline and security, why is it that these policies target largely high-poverty, urban, majority non-white schools instead of the schools actually more likely to experience a mass shooting event, according to the dominate narrative?

Contending Narratives, Disparate Costs 

A quick response is to say that the high security environments of urban schools prevents mass shooting from happening there in the first place, but this argument lacks causal evidence. City Lab borrows data from a Mother Jones compiled data on mass shooting events to show that, more or less, these events are likely to occur in any town– urban to rural, poor to rich, and across racial and ethnic groups. Mass shooting events though are perpetrated by mostly white males. Gun violence occurs in urban school neighborhoods as well and the Washington Post explains how students at urban schools experience this violence at and near school, and their feelings of frustration and solidarity following Parkland.

But, these facts illuminate two contending narratives about school violence and safety.  On one hand, the lone white male perpetrator. On the other hand, mostly non-white urban schools experiencing violence from interpersonal and gang-related conflict.

These two narratives are both based in reality, but poorer urban mostly non-white schools are those most likely to actually implement zero-tolerance discipline policies alongside carceral school security measures. A meta-analysis of the impact of metal detectors in schools notes that “Although metal detectors may hold an appeal because they are viewed as a quick and visible solution to school violence, schools must also weigh evidence of effectiveness and cost when determining which youth violence prevention strategies to employ. Given the lack of clear evidence of the effectiveness of metal detectors in preventing violence as well as the significant cost, schools may wish to either incorporate metal detectors as one component of a comprehensive program, or explore alternate strategies.” (Hankin et al. 2011)

Quick, and visible. Effective at preventing violence? Unclear. What we do know however is that carceral school environments negatively impact students perceptions of themselves, including internalizing messages of criminality, and of their schools. Metal detectors are also expensive, and students may perceive this as a misallocation of resources. Unequal City documents both reaction in Chicago Public Schools.  Internalizing messages of criminality, perceptions of injustice, mistrust in school authority, and so on: these are costs imposed on students attending schools with carceral environments, all of which can negatively impact students’ long-term educational trajectory, and therefore their economic and social outcomes. Exploring “alternate strategies” would be a better approach, including investment in counseling, neighborhood resources, trauma-informed practices, and restorative justice programs for students to feel safe, cared for, and secure in their schools and communities.