Statistics and the Johns Hopkins Gun Study

This week, the Johns Hopkins University Bloomberg School of Public Health alongside the Johns Hopkins Center for Gun Policy and Research announced the forthcoming release of a study linking the repeal of Missouri’s Permit to Purchase (PTP) handgun law with an increase in the state’s murder rate. The report, set to be released in a coming issue of Journal of Urban Health, claims that since the 1921 law’s repeal in 2007 the homicide rate in Missouri has increased and that the absence of such a law has accounted for an additional 55 to 63 murders per year. Given this claim, I had to dig into the issue.

First of all, it should be noted that the study claims to ‘control’ for factors such as poverty, incarceration, and unemployment. Unfortunately given the report’s unreleased status, it is impossible to determine exactly what is meant by this. As a society, we cannot even fully agree on how to quantify these issues, much less fully account for them statistically. We must also remember that firearms policies do not exist in a vaccuum. That is, a combination of several factors can influence crime rates in a given area and how these rates might change over time can also be affected by the same variables. To presume that anyone fully understands the causes of crime well enough to control for all but firearms policy is an extremely lofty assumption.

With the above disclaimer and understanding we do not currently know the study’s methodology, it is still possible to address the study’s claim head on. As I said before, the study asserts that murder rates increased after the PTP law’s 2007 repeal, to a tune of more than 55 additional murders per year. Unfortunately for this ‘unbiased study’, the truth does not seem to support the school’s claim. The table below was sourced from the FBI’s Uniform Crime Reports. As many readers are likely to realize, the UCR are generally considered the most accurate reports of crime data within the United States.

FBI's UCR data for Missouri

FBI’s UCR data for Missouri

According to the UCR, the homicide rates appear to have remained remarkably steady within Missouri since 2000. In fact, total homicides have shown no notable increase while the population has increased by nearly 500,000 people over this period. Based on this knowledge, it would be reasonable to argue that Missouri homicide rates have actually decreased over the past several years. However, that is not the aim of our discussion today.

A deeper dive into the statistics reveals some interesting points about violence in Missouri. Though the PTP law dates back to 1921, FBI UCR data only goes back to 1960. Still this should give us a good basis for comparison. With this in mind, I went back and regenerated the table to fetch entries from 1960 to 2012. Based on this data, I was able to determine that the average homicide rate from 1960 to 2007 (the year the law was repealed) was 8.11 incidents per 100,000 residents. This time period exhibited a standard deviation of 1.96. The fact that a single standard deviation makes up almost one-fourth of the total means that the homicide rate in Missouri fluctuates considerably.

After looking at the period prior to the law’s repeal, I turned my attention to the five years following the end of the law. From 2008 through 2012, the homicide rate in Missouri averaged 6.76 incidents per 100,000 residents. Contrary to the study’s claim, the homicide rate has actually dropped following the law’s repeal. A link to the Excel file with the full data can be found below.

https://drive.google.com/file/d/0BwpMFLaRK-kFRUs4YnMzTDdPU3M/edit?usp=sharing

After analyzing the UCR statistics, I felt that it would still be important to try to understand how the study came to such a conclusion. Without having access to the study, I approached the issue with the null hypothesis (working assumption) that the law’s repeal had no effect on crime rates. This was chosen because the alternative hypothesis (that the law did reduce homicide rates) is much more difficult to assess, given the lack of post-repeal data. I will spare readers a discussion on p-values in this article as I feel the results of our analysis are telling enough and we can come to a conclusion without invoking further statistical evaluation. The post-repeal data exhibits an average of 1.36 fewer incidents per 100,000 residents each year. The fact that this difference is not only within a single standard deviation of the pre-repeal mean, but also lower than the mean for the period prior to the PTP law’s repeal is notable. Using typical research practices, in order to reject the null hypothesis, post-repeal homicide rates would need to average at least two standard deviations higher than the pre-repeal mean. Based on the information available, it seems that the Johns Hopkins researchers fell prey to a remarkably egregious type I error (improperly rejecting the null hypothesis), damaged the data in ‘controlling’ for other factors, used an improper null hypothesis to begin with (perhaps by using our alternative hypothesis), or some combination of these factors.

Furthermore, Missouri State Highway Patrol data supports our conclusions. The graph featured below was sourced from the MSHP’s Statistical Analysis Center. As we can see, the number of murder offenses committed each year has remained very steady over the past several years, though arrest numbers have varied wildly. As we can see, 2010 was a particularly bad year for policing in Missouri.

MSHP murders and arrests by year

MSHP murders and arrests by year

Some readers may be asking, “why are the numbers in the UCR and MSHP data different?” In short, this is likely because the underlying reporting used to generate the data can often be incomplete or inconsistent. As such, final numbers are close approximations of crime levels for a given year. These numbers should not be taken for their precision, but should be viewed as good estimates. This makes the Johns Hopkins claim even more dubious given the level of precision that the study seems to assume.

MSHP robberies and arrests by year. More firearms equal fewer robberies? We cannot be sure, but interesting nonetheless.

MSHP robberies and arrests by year. More firearms equal fewer robberies? We cannot be sure, but interesting nonetheless.

So what does all of this mean? According to former University of Chicago and Yale University professor, Dr. John Lott, the study’s conclusion does not mean much at all. In his critique, Dr. Lott highlights that the researchers at Johns Hopkins had to cherry pick data to fit their conclusion. In other states, the results of such licensing schemes have yielded much different results. One example that Dr. Lott highlights is Massachusetts, where just prior to the law’s passage, the state’s murder rate was less than one-third the national average. In just 10 years after the law took effect, the state’s murder rate nearly doubled as compared to the national average.

Murder rates in Massachusetts compared to national average

Murder rates in Massachusetts compared to national average

Lastly, we have to consider the source of this ‘study’. Johns Hopkins is a school that has deservedly gained substantial attention worldwide for its remarkable medical programs. The university is undoubtedly at the top of many aspiring doctors and researchers’ lists. Still, we must not let this reputation cloud our common sense and good judgment when addressing this misguided piece set to be published by Johns Hopkins’ Bloomberg School of Public Health.

As some readers have probably realized by now, the Bloomberg aspect of the school’s name is indeed the result of a $1.1 billion donation from fanatical anti-gun crusader and former New York City mayor, Michael Bloomberg. Last year, New York’s most famous control freak and junior despot made this massive donation so that the university could make numerous campus improvements, including the construction of a school of public health.  According to a NY Times report, $250 million of the donation was to be used to hire 50 new researchers. Lead author of this most recent study, Daniel Webster, has even published a book, Reducing Gun Violence in America, with a foreword by Michael Bloomberg. To say the study is ‘unbiased’ is incredibly disingenuous.

While the study set to be released by Johns Hopkins Bloomberg School of Public Health is certain to present itself under the guise of unbiased professionalism, a critical analysis of the study reveals serious holes in its claims. The assertions made by the study simply do not reflect reality in Missouri and are based on some very questionable statistics. Furthermore, the school’s credibility as a policy research center is completely destroyed by its relationship with fanatic Michael Bloomberg. While it will be worthwhile to read through the whole study, readers should embark upon this task with extreme skepticism.

Study announcement: http://www.eurekalert.org/pub_releases/2014-02/jhub-rom021414.php

UCR data: http://www.ucrdatatool.gov/Search/Crime/State/RunCrimeStatebyState.cfm

MSHP data: http://www.mshp.dps.missouri.gov/MSHPWeb/SAC/crime_data_violent_crime_960grid.html

Lott on study: http://crimepreventionresearchcenter.org/2014/02/what-does-missouri-show-about-the-benefits-from-universal-background-checks-the-forthcoming-journal-of-urban-health-study-by-the-bloomberg-school-of-public-health/

Lott on Massachusetts: http://johnrlott.blogspot.com/2013/02/massachusetts-murder-rate-has-risen.html

NY Times on donation: http://www.nytimes.com/2013/01/27/nyregion/at-1-1-billion-bloomberg-is-top-university-donor-in-us.html?pagewanted=all&_r=1&

Webster book: http://www.amazon.com/Reducing-Gun-Violence-America-Informing/dp/1421411105#reader_1421411105

Advertisements

4 thoughts on “Statistics and the Johns Hopkins Gun Study

  1. As my habit, I like to go to source material. The John Hopkins “source” is actually an *editorial* on the manuscript that is not yet published. Almost by definition, editorial reflects/advocates an agenda, so I’m going to take anything with a grain of salt at this point. JH did poor service for their authors, though, because now the manuscripts reception is going to be biased by an agenda driven editorial. You provide a good analysis of this.

    There are a couple of additional questions that I do hope the paper addresses. I’ve seen a graph relating to the murder rates reported and indeed it does show the increase claimed. However, that data also has a more marked increase in the same period prior to the repeal of the PTP law. How did the address whether the repeal actually resulted in a reduction in the rate of increase? If this isn’t addressed explicitly, is this noted as a limitation of the study or a question for further investigation?

    The other question the preliminary reporting brings to mind is whether they look at the deltas for gun theft in the same period? I.e. was there an increase or decrease in gun theft associated with the repeal of the PTP law. If so, how did that correlate with the changes in murder rates?

    • Raul,
      Thank you for your comment. You are correct that many questions remain unanswered when it comes to this report. My hope is that the public release of the study will shed some light on these. Admittedly, I anticipate that the released study will confirm my initial suspicions. Beyond the points I addressed in my post, I expect we will see JHU come to the conclusion that the repeal of the PTP law may have allowed for a small increase in gun-related crime and that the institution has mislabeled this as an increase in overall crime. A superficial analysis of such a claim could lead them, whether biased or ignorant, to claim that homicide rates increased, without considering criminals’ tendency to commit the crime with whatever tool is available. See my discussion on Australian gun control for an example of how criminals made a near 1:1 switch to bladed weapons upon the ban there.

      We will see what comes of this. I plan to readdress the issue at that time, but for now things certainly do not look all that legitimate.

  2. Have you reviewed/renewed your analysis of the final report?
    http://www.cbsnews.com/htdocs/pdf/missouri_background_checks.pdf
    I love math, but don’t have any statistics skills. I’m concerned about the Missouri paper and of course the publicity, but it’s hard to discuss without going into all the technojargon. One obvious point is that the numbers are still very small. A 25% increase in a tiny number is still a tiny number. The Mayors Against Guns group has another handout comparing private-sale-background-check to non psbc, with impressive results, but again small numbers. Like to hear an expert analysis on that as well.
    http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&ved=0CCcQFjAB&url=http%3A%2F%2Fwagunresponsibility.org%2Fwp-content%2Fuploads%2F2014%2F06%2F9.2013-Background-Checks-Reduce-Crime-and-Save-Lives.pdf&ei=4-wWVOXHA47hsATT_oLgDQ&usg=AFQjCNEtDTvR5QB2xMSMfllE-vN0UG76nQ&bvm=bv.75097201,d.cWc

  3. Right here is the perfect web site for everyone who wants to find out about this topic.

    You realize a whole lot its almost tough to argue
    with you (not that I personally would want to…HaHa). You definitely put
    a fresh spin on a topic that has been discussed for
    many years. Excellent stuff, just excellent!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s