Text by: Rodrigo Serrano-Berthet *

I would like to start by making a statement that may sound a bit controversial:
 

Those in charge of decision-making regarding security and justice in Latin America and the Caribbean, in general, have not shown much interest in knowing whether their policies work or not, whether they are effective or not, in preventing and reducing crime, violence, and insecurity that plague the region. 
 

I am not saying they are not interested, but that they have not shown it. If crime goes down, it is due to their policies. If crime goes up, it is due to factors beyond their control. The opposition presents the opposite arguments. The quality of the public debate on the effectiveness of security policies does not usually go much beyond this level of sophistication. And as a consequence, in most cases we do not know if our interventions have any impact on crime reduction. I am talking about the average, of course; there are always exceptions. 

There are several reasons that help to understand this situation, but before going into that topic I would like to explain what I base this statement on.  
 

How to measure governmental interest in the effectiveness of security and justice programs?

Without claiming to be exhaustive, I propose three "rough" measures of what would be desirable to find.
 

1) Impact evaluations (quantity, quality, promotion, use)

If there were interest in effectiveness, the region would have thousands of studies evaluating whether it was possible to attribute any effect on robbery, gender violence, or whatever the program sought to impact, to "program x" or "measure z". A significant percentage of these studies would be of high scientific rigor in terms of causal inference, and a high proportion of them would have been supported by the government. Over time, we would observe that the programs that worked gained scale and those that did not work were extinguished. 

That would be the ideal. Now let's look at reality.

To create the Evidence Bank, at the IDB we were doing an exhaustive search for impact evaluations in the region. So far we have found about 200 studies, of which about 60 meet scientific rigor criteria to infer causality. 

This is not negligible, but it is a small number, in my opinion, if we consider that the region has 33 countries that for more than 20 years have been implementing all kinds of security programs at the national level and in thousands of subnational governments. Of the studies we found, only a minority were financed directly or indirectly by the governments of the region. 
 

2) Evidence-informed program design

If there were interest in effectiveness, every time a new program had to be designed or a new policy decision made, the decision-maker would have sought to be informed by the best available scientific evidence on the causes of the problem they wanted to address and on the solutions that had worked to face that problem, would have adapted them to their context, and would have designed agile mechanisms for iterative learning. We would see this reflected in the program design documents and in its monitoring and evaluation system. 

Now let's look at reality.

A few years ago we did a study that, although partial and imperfect, gives us some clues. In six countries of the region we analyzed the design of 283 citizen security programs and found that only 8% included information on the evidence of effectiveness of the proposed type of intervention.

Access the full study here.

3) Organizational capacities for learning (Criminal Analysis Units)

If there were interest in effectiveness, the organizations of the security and justice system (police agencies, prosecutor's offices, etc.) would have robust Criminal Analysis Units, with the capacity to analyze the patterns and causes of the problems they seek to solve, as well as to test solutions and measure the efficiency and effectiveness of alternative solutions in real time. 

I am not aware of studies on this, but based on the experience of the IDB's citizen security team, we can say that, while the number of Analysis Units is growing, in general they are disconnected from the decision-making process. The focus is on the production of descriptive statistical reports, and in very few cases are there analytical products oriented towards management decision-making, much less towards measuring the effectiveness of the decisions made. 
 

What barriers hinder interest in effectiveness in security and justice?

In another blog I wondered why security and justice policies in our region "were oriented less by science and more by beliefs, copying what seems popular, or what gives more political gain, giving free rein to ideologies and prejudices of all kinds." 

There I focused on one of the barriers: the difficulty that decision-makers have in accessing science, the already systematized knowledge about what has worked and what has not. The Evidence Platform and the Evidence Bank seek to contribute to closing this access gap. 

But that is not the only barrier, nor the most important one.

The most important barrier that explains the lack of interest in effectiveness is the low importance given to crime prevention in security and justice policies.
 

The "disadvantages" of prevention

The security and justice system has two main functions: to enforce the law (or repress crime), and to prevent the occurrence of crime. Of these two functions, repression is the easiest and most natural to fulfill and, therefore, the one that usually concentrates all the energy of the institutions. Police agencies in the region, for example, dedicate most of their time to reactive, crime response activities, such as attending emergency calls, arrests, or criminal investigations. 

Defining their institutional mission around these control activities has several advantages: it allows a police agency to address the most immediate citizen demands, the requests from politicians for visible actions, and to do so without having to depend on anyone other than the organization itself to achieve it. It gives them an "argument" to ignore questions about how effective these actions are in preventing crime. An arrest, an answered emergency call, or a concluded criminal investigation have intrinsic value, and are an essential part of the police mission.

And why is it "good" not to have to answer questions about the effectiveness of preventing crime? Because prevention is difficult, can take time, does not necessarily generate visible actions, and requires the cooperation of others. And the risk of failure is greater. 

I am talking about prevention done seriously, not simply putting more police to patrol or visit schools giving talks on drug prevention, assuming they will be effective. Prevention, taken seriously, requires asking about effectiveness, while "enforcing the law" manages to justify itself based on efficiency measures.

The problem is that, to know if an activity had a preventive effect, some type of impact evaluation is required: it is not enough to show the number of activities or products carried out. And these impact evaluations carry a high risk for the organization: showing that what they did did not serve to prevent crime or, worse, that they left society with more problems than it had before.

In summary, the social value of law enforcement can be measured through "products" that are under the control of the police. The social value of prevention can only be measured through "results" that, inevitably, will be partly outside the control of the police.  

So, as an organization, why am I going to seriously worry about prevention if I am much safer and more comfortable limiting myself to my role of "enforcing the law"? 
 

The social value of prevention

The main reason to do so is that any citizen would prefer a thousand times more not to be a victim of a crime than to have their emergency call answered, or the criminal arrested or investigated. From the point of view of value to society, reducing and preventing crime is much more important than reacting to it. 

The most modern police forces in the world have already understood this, and give prevention a central role in their work. They approach each of their tasks (including control tasks) with a preventive logic. This obviously includes the police deployment in the territory, where the effect on crime prevention can be zero (if done poorly) or imply a 20% reduction in crime if hot spots policing strategies are adopted.

But even in repressive work they have a preventive approach and, instead of investigating crimes case by case, purely from a reactive logic, they analyze crime patterns, identify underlying common causes, define what the police should do, and act proactively on those causes.

Making this change requires leaving the comfort zone of the reactive model, and starting to ask the question about what is the most effective way to achieve the most socially valuable results.

To do this, we need to make prevention more attractive to police organizations, for citizens to value and demand it, for politicians to see the advantages of promoting it, for the police to be less afraid of failing, and to become organizations oriented towards continuous learning and the search for results. 

And what we say for the police is also valid for the other organizations in the criminal justice system.

Getting security and justice organizations to identify with prevention as the central focus of their work, to have the institutional courage to do so, is perhaps the most important objective we can have for those of us who seek to increase the effectiveness of security policies by increasing interest in evidence-based policies. 

From the Evidence Platform, we modestly seek to help this paradigm shift. By disseminating preventive models that have worked and criticizing those that have not worked (but remain popular), and by strengthening the nascent community of practice of political leaders, security professionals, representatives of the academic sector, and activists committed to prevention and effectiveness in development.

Write to us with your ideas on these issues, on what is being done and what more could be done to increase the interest of authorities in crime prevention and in evidence-based policies.

* Rodrigo Serrano-Berthet is a lead citizen security specialist at the IDB.