Assessing the Effectiveness of Organized Crime Control Strategies: A Review of the Literature

2. Methodology


Literature reviews tend to contain, at most, very brief sections describing the manner in which documents have been located and specifying the criteria serving as the basis for their inclusion. However, the process of identifying and selecting materials for this review was challenging due to the breadth and complexity of OC, as well as the dearth of evaluative studies that fit the conventional mould.

The terrain of OC is so broad—covering such a wide range of illicit activities—that the pool of materials through which one must sort is daunting. As an illustration, an initial search of several electronic databases yielded more than 800 documents covering the relatively short period from 1990-2003. One must therefore engage in a major selection process to reduce the volume of documents to a manageable number.

Apart from the sheer volume of materials, there are differences of opinion regarding the definition of organized crime and, hence, the scope of activities subsumed within the concept. Also, the document selection process is rendered difficult by the fact that very few studies have rigorously evaluated OC control strategies through conventional research designs. For example, if one was to review treatment programs for men who batter their spouses, there is a finite pool of several dozen experimental and quasi-experimental studies evaluating the effectiveness of these programs.

In the context of OC, such designs would involve the careful monitoring of a control strategy (e.g., a law enforcement initiative), either from the outset or after the fact, to determine whether statistically significant reductions in specified illicit activities or other changes have occurred following their implementation or in relation to comparable jurisdictions in which such initiatives have not been adopted. The use of such designs is virtually non-existent in assessments of the impact of OC control strategies. In fact, Maltz (1990:15) argues that the classic experimental designs cannot be used in evaluating the success of OC control efforts, "as this would be tantamount to advertising that the "control" jurisdictions are (and will remain) free of federal enforcement efforts."

That highly rigorous assessments are few in number is a conclusion drawn not only by this author but in some of the most authoritative texts on OC. For example, Reuter (1994:91) asserts that the OC literature as a whole has failed to attract much scholarly attention: "It would be difficult to identify as many as half a dozen books that report major research findings, or even that many significant articles."

Martens (1983; cited in Beare, 1996:35) adds that, "The field of organized crime research, to say it modestly, suffers from ‘intellectual atrophy’. Little is written that deserves our attention and that which is written often does not reflect reality… Particularly when it comes to addressing public policy issues with respect to organized crime control, the field could use a healthy dose of ‘real world’ experiences."

The problem may be even more acute with regard to evaluations of OC control strategies. On the strategy of prosecuting and incarcerating the heads of criminal organizations, for example, the US President’s Commission on Organized Crime (1986:205) notes that no rigorous assessments or evaluations of the "headhunting" strategy are available for inspection.

On the impact of the regulation of large-scale, illegal gambling by law enforcement agencies, Reuter (1984:45) writes, "Unfortunately, it is not possible to test these hypotheses… Analysis of differences between cities in which strike forces were present and those in which they were not has never been carried out."

On anti-money laundering measures, Levi (2002: 188) states, "Given the political and social importance of this area, the absence of evaluation research on it and organized crime in general is remarkable." Later in the same article, Levi adds that, "Demonstrable impact on crime and even on crime detection remains harder to identify, however. Methodological problems in linking causes to effects mean that there are few defensible positive findings about the direct, short-term impact of money-laundering reporting on prosecutions and on confiscation" (p.190).

Also on money laundering, Walther (1994:9) suggests that research in this area is still at a nascent stage:

Forfeiture and money laundering laws undoubtedly provide for harsh sanctions in the individual case, but do they deter? It appears that a lot of empirical and theoretical research work is awaiting criminologists and jurists in this area. The question of deterrence undoubtedly raises a set of rather complex issues. At the outset, however, it would be interesting to know about the ratio between the amount of profits that are in criminal enterprises, and the proportions that so far have been successfully taken out of it. Similarly, we may wish to look at data about the ratio between the number of investigative measures, the number of arrests that have been effectuated under the new and broadly defined criminal statutes [in the United States], and the number of subsequent prosecutions and convictions.

Fyfe and McKay (2000:280) assert that witness protection programmes, too, have been the subject of little evaluative research: "Despite nearly thirty years of formal witness protection programmes operating in the US, research in this area is in its infancy."

Lyman and Potter (1997:434), in a leading textbook on OC, add their voice to the lack of routine data collection in this area, asserting that "The [US] government keeps no comprehensive statistics on how many organized criminals have been arrested, convicted, and incarcerated."

The lack of routine data collection and rigorous evaluations in this area necessitate some compromises with regard to the rigor one could reasonably expect from works to be included in the present analysis. Some subjectivity was unavoidable in this exercise. Before discussing the criteria for inclusion, let us turn to the parameters of the search strategy.

2.1 The Search String

Time Frame
A preliminary scan of materials indicated that most assessments of OC control strategies have tended to appear since 1980. The focus of our search was from that year onward, although seminal works appearing prior to 1980 were included in this review.
While a fair amount of material on OC has been published in languages other than English and French, this review is confined to works published in Canada’s two official languages.
Scope of the Search
The search was conducted with the help of several electronic databases. The databases searched included Sociofile (1980-2003), Current Contents (1993-2003), and Quicklaw (1990-2003). Current Contents affords coverage just from 1993 onward and the search with Quicklaw yielded so little from 1990 to the present that the search was not expanded to the 1980s. As OC was the critical unit of analysis in this review, the key words used in the electronic searches were: "Organiz(s)ed Crime", "Criminal Organization", "Transnational Crime", and "Criminal Network(s)".

Bibliographies provided by various Web sites and in major textbooks on OC also proved to be rich sources of information. Furthermore, the bibliographies contained in the documents uncovered by the electronic searches were scanned for additional material.

2.2 Document Selection Criteria

Materials examined in this review included books, articles published in academic journals, government or law enforcement publications, reports produced by research institutes, and newspaper articles. While hundreds of works explicitly or implicitly relate to OC, the following requirements needed to be met before a document was included in the analysis:

  1. The document had to provide some assessment or evaluation of a strategy, approach, or program designed to control OC or it had to deal with methodological issues in these assessments. While advanced research designs were not required (see above), simple assertions purely of an anecdotal nature were not sufficient to include a document. Some empirical evidence was required to support the author’s contention regarding the impact of a control strategy. Also included in the analysis were documents dealing with definitional and methodological issues bearing on evaluations in this area.
  2. Organized crime had to be an important theme in the work. Many publications deal with activities in which criminal organizations may be involved—e.g., racketeering, drug trafficking, money laundering, and prostitution. In many instances, however, the focus is on the activity and there is little connection made with OC. If all these works were included, every initiative dealing with every independent operator would be included. In the case of drug trafficking, for example, every measure to curb demand, including psychotherapy provided to individuals, would be included because such interventions may ultimately reduce OC’s client base. As OC was the unit of analysis in this review, more direct and explicit connections had to be made between the control measure and the impact on criminal organizations. While the decision to include or exclude works was often clear, there was no escaping some subjectivity in the selection of documents.

2.3 Format of the Analysis

Due to the dearth of more rigorous evaluations of the experimental type, the analysis did not lend itself to the tallying of research findings or the amalgamation of research findings in the form of a meta-analysis. The review therefore assumed a more conventional format, whereby the evidence derived from assessments of each OC control strategy was presented and weighed. At the end, the aim was to ascertain the merits of different control strategies, while maximizing impartiality and carefully assessing the credibility of the evidence presented.

Date modified: