How Flawed Are National Policing Statistics?

Police statistics are used to sway major policy decisions and channel billions of dollars in funding. The statistics can boost or undermine police legitimacy—an important predictor of crime. And they can be used by the public to force police departments to change their practices.

But the numbers are often incomplete or misleading.

Virtually all common statistics that deal with police work should only be used with major caveats, or at least with sufficient explanation, experts say.

“Data serves to researchers or to scientists in the same utility that a thermometer would serve a medical doctor,” said Alex del Carmen, professor and executive director of Tarleton State University’s criminology department. “No medical doctor using a thermometer will say that you’re dying from cancer if you read outside of 98.6.”

Data Collection

Most of the national police statistics are collected by the FBI and the Department of Justice (DOJ) through its Bureau of Justice Statistics (BJS).

But it’s voluntary for police departments to submit their data, as there are laws that prevent the federal government from interfering in states’ affairs.

Police departments serving 90 percent of the population do submit their data, which makes the FBI’s Uniform Crime Reporting Program the most comprehensive collection of statistical reports on policing.

However, the data is still incomplete. Cleveland, for example, didn’t report its 2015 numbers because of a problem with a new data collection system, yet the city has a violent crime rate more than three-and-a-half times the national average.

Furthermore, the FBI data only counts the most serious offense in every case. “If a burglary and a rape took place in the same incident, only the rape would be reported,” said del Carmen.

And even further, he noted, the FBI data only contains reported crimes, while the majority of crime goes unreported.

The BJS calls 90,000 households every year and asks people if they have been the victim of a crime in the past year.

The survey found that close to two-thirds of property crimes (burglary, theft, car theft) and almost half of violent crimes (rape, robbery, assault) went unreported in 2014.

According to the BJS survey, one in 50 people (age 12 or older) reported being a victim of a violent crime in 2014, whereas the FBI reported one in 267 people was a victim, based on 2014 crime rates.

The BJS survey is also limited: It excludes children under 12, the homeless, people in nursing homes, jails, and prisons, and murder victims.

Data on Police-Involved Homicides by Source

Police Shootings

One of the most sensitive FBI statistics—on police-involved shootings—is also one of the least reliable.

Police departments reported fewer than half of the police-involved killings that likely took place in the years 2003 to 2009 and 2011, according to an analysis by RTI International, a research nonprofit, conducted for the Justice Department and released last year.

The FBI reported that police officers killed 404 people in 2011. RTI estimated the actual number was somewhere between 998 and 1,176.

Police departments may not report police homicides because of gaps in training, but it may also be that police resist the data collection.

“It draws negative public attention to police departments,” said Melissa Hamilton, visiting criminal law scholar at the University of Houston.

“[A] lot of times, the public is willing to assume now that police officers acted incorrectly.”

Experts have also cautioned against relying too much on open-source data, sometimes assembled by publications that have tried to fill the gap. The Washington Post and The Guardian newspapers started compiling their own tallies in 2015 based on news reports and other online sources.

“It’s really difficult to know whether those are accurate or not,” said David Klinger, professor of criminology and criminal justice at University of Missouri–St. Louis.

If, for example, media pay more attention to certain types of cases, the data would show a bias in that direction, Klinger said.

“I just think it’s better to be very, very cautious,” he said. “Particularly in [regards to] something like this where the stakes are so high.”

——————

How to Read the FBI’s National Police Statistics

Although the FBI data on the following crimes are the most comprehensive, they are incomplete. This is a guide for how to adjust data to account for unreported crime and underreporting to the FBI.

Murder
Add up to 10 percent

Aggravated Assault
Add about 50 percent

Burglary
Add about 55 percent

Robbery
Add about 90 percent

Police-Involved Killing
Add 100 percent to 200 percent

Source: FBI Unified Crime Reporting (data for 2014); BJS National Crime Victimization Survey (data for 2014); RTI International, Arrest-Related Deaths Program Assessment, 2015

—————

Initiatives

In 2014, Congress passed the Death in Custody Reporting Act (DCRA), which will require state and federal law enforcement agencies to submit data to the Justice Department about people who died during interactions with law enforcement or in their custody. The department can withhold federal funding from states that don’t comply.

The initiative seems to be a ways off from being implemented, though. “Last week, the first public comment period closed, with several thousand comments received,” the Justice Department announced on Oct. 13. “The department is currently reviewing those comments and it plans to issue an updated proposal in the near future.”

Luis Fernandez, professor criminology and criminal justice at Northern Arizona University, expects mixed results from the initiative.

“My guess is that there will be some departments and some places and some chiefs that are going to come in very strongly and try to change the culture and there are going to be some places that are going to resist that change pretty drastically,” he said.

If the Justice Department manages to collect sufficiently complete data, it may open an opportunity for civil rights lawsuits, according to Fred Brewington, who has been a civil rights attorney for 30 years in New York and New Jersey.

Brewington has seen that claims of police misconduct have become more believable in recent years thanks to the proliferation of videos of some incidents.

Police statistics on the use of force can be very useful in civil rights cases, Brewington said, because they can be analyzed for signs that a police department engages in a pattern of abuse or discrimination. A court can then force the department to change its practices.

For example, U.S. District Judge Shira Scheindlin ruled the New York City Police Department used unconstitutionally its practice of stopping and searching people, known as stop-and-frisk. A statistical analysis of the stop and frisk data played a significant role in her ruling.

“It required millions and millions of evaluations of reported data that became available only because the city of New York started to collect that data,” Brewington said.

Still, experts have cautioned against reading too much into law enforcement numbers.

Harvard economist Roland G. Fryer conducted detailed analysis of the NYPD stop-and-frisk data as well as police shootings data in several large cities. His analysis showed signs of discrimination in the stops and frisks, yet he also stated that “we have no definitive proof of discrimination” just from looking at the numbers, because race can’t be assigned at random.

On the other hand, not collecting the data or collecting it poorly may hurt police credibility.

The lack of national data on police killings itself strengthens the argument that the country has a problem with holding police accountable, according to Fernandez.

The key seems to be to understand the story beyond the numbers.

“We want the data. Just we need to understand that the data is within much larger context,” Fernandez said.