The Effectiveness of Using Mediation in Selected Civil Law Disputes: A Meta-Analysis
Only 28 of the 37 unique studies of a specific mediation program provided data that allowed us to generate effect size estimates, while 34 of the 37 unique studies provided data that provided us with averages for other outcome measures. In all, 59 effect sizes and 97 other averages of outcome measures were calculated.
|Outcome Measure||Number of Effect Size Estimates: Unique Programs or Sites||Number of Effect Size Estimates: Comparison Group Disaggregated|
|Satisfaction with Outcome||15||22|
|Satisfaction with Process||11||17|
|Comply with Agreement||5||5|
Studies commonly collected information on the impact that the mediation program had on case processing. However, these variables are captured not as percentages, but as whole number counts (e.g., 5.3 hours), which do not allow for the calculation of effect size. These measures are not ‘outcome’ measures in the sense that they measure a final result of the application of the mediation program on a temporally independent variable, but they can be considered ‘outcome’ measures in that they capture the results of the impact of the mediation program on considerations directly related to the reason mediation programs are initiated. Thus, the measures in Table 2, therefore, are not ‘outcome measures’ in the typical sense used in meta-analyses of treatment programs. However, as they do objectively gauge the impact of mediation programs they have been included in this report through an analysis of averages.
|Measure||Number of Averages: Unique Programs or Sites||Number of Averages: Comparison Groups Disaggregated|
|Staff Hours Saved||13||13|
|Number of Hearings||6||9|
|Number of Pre-Trial Conferences||11||17|
|Number of Motions||12||12|
The studies were conducted between 1980 and 2004 with a median year of 1995. Since data analysis and dissemination take some time, the median date for the publication of results was 1997. Most of the studies were from the United States (68%) and most were published by governments (62%).
The most rigourous method for setting up a study is to use a random sample as a comparison group. This method was, indeed, used most frequently (43%) in the studies. Surprisingly, our sample of studies only included a few studies using the relatively simple method of a pre-post project implementation for comparison purposes (5%). (The possible reasons for this are discussed in ‘Future Research’, section 5.1 of this report.)
|United States||25 (67.6)|
|United Kingdom||1 (2.7)|
|Non-government Organization||5 (13.5)|
|Random sample||16 (43.2)|
|Comparison group||7 (18.9)|
|Matched comparison group||12 (32.4)|
|Pre/post design||2 ( 5.4)|
The mediation programs being examined in most studies were ones where tort and contract cases were allowed (60%) and the area of law covered by the mediation program was ‘mixed’ (57%) making it difficult to ascertain if the area of law or type of litigation impacted upon the outcome measures.
As can be seen in Table 4, many of the studies featuring facilitative mediation techniques in the mediation program (35%), but not many of the mediation programs examined in the collected studies (16%), were mandatory mediation programs. A greater number were mediation programs where mediation could be ordered by the court (35%). Although it may appear that mandatory programs and programs ordered by the court might be lumped together for the purposes of analysis, they cannot. While the level of coercion for ‘mandatory’ and ‘ordered by the court’ is similar, the mandatory programs captures all of the cases that met the mediation program criteria while the ordered by the court only includes a skewed sub-set of cases, thereby precluding aggregating the categories for the purposes of analysis.
|Tort & Contract||22 (59.5)|
|Not noted or unclear||1 ( 2.7)|
|Business or Commercial||3 ( 8.1)|
|Personal Injury||4 (10.8)|
|Small Claims||4 (10.8)|
|Other||1 ( 2.7)|
|Party driven||1 ( 2.7)|
|Ordered by court||13 (35.1)|
|Mixed||1 ( 2.7)|
|Not noted||14 (37.8)|
|Complexity||3 ( 8.1)|
|Not noted||20 (54.0)|
|Business, Organization or NGO||13 (35.1)|
|Government||3 ( 8.1)|
|Not noted or other||9 (24.3)|
|Business, Organization or NGO||25 (67.6)|
|Not noted or other||4 (10.8)|
Experienced mediators indicate that the characteristics of the individual mediator are an important factor in the outcome of cases (Gordon Mamen, personal communication). Thus, a number of variables characterizing the mediator are often presented in studies of mediation programs.
The majority of the studies in this meta-analysis examined mediation programs where the mediator was paid (81%) and had legal training as either a lawyer or paralegal (87%). Most mediation programs did not allow the litigants any freedom in selecting their mediator (57%). Interestingly, who paid the cost of the mediation was fairly evenly split between litigants (30%) and the government, program or court (35%). The remaining categories of mixed (8%) and not noted (27%) could possibly represent a similar distribution.
|Paid (Noted)||16 (43.2)|
|Paid (Implied)||14 (37.8)|
|Paid (Both)||30 (81.1)|
|Volunteer (Noted)||6 (16.2)|
|Not noted or unknown||1 ( 2.7)|
|Yes (Lawyer or Paralegal)||16 (43.2)|
|Yes (Lawyer or Paralegal Implied)||16 (43.2)|
|Yes (Both)||32 (86.5)|
|No||3 ( 8.1)|
|Not noted or unknown||2 ( 5.4)|
|Not noted||5 (13.5)|
|Before||2 ( 5.4)|
|During||2 ( 5.4)|
|Not noted||29 (78.4)|
|Government, Program or Court||13 (35.1)|
|Mixed||3 ( 8.1)|
|Not noted||10 (27.0)|
A number of other mediation programs measured process-related outcomes. The average value that was under dispute in the mediation cases was just over $74,000 Canadian. The average fees paid to the mediator per case were around $850 Canadian. The history of litigiousness for those entering mediation was not reported very often (k=2). The number of months that it usually took from the time a case was referred to the mediation program until the first mediation was reported fairly often in the studies (k=24). The average length of time was nearly 5 months.
|Variable||Mean (k) |
|Claim Value||C$ 74,070.50 (12)|
|Average Fee||C$ 859.90 (18)|
|Preparation Time||1.39 hours (13)|
|Litigiousness of Mediation Group||45.2 % with a history ( 2)|
|Referral-to-Meeting||4.90 months (24)|
The kind of group against which the mediation program is being compared is important. The best comparison group would be those cases which were eligible for the mediation program, but litigated instead. This group represents a substantial proportion of all comparison groups (30%). The most common comparison groups were adjudication (32%).
|Dropouts ||1 ( 2.7)|
|Litigation (Eligible)||11 (29.7)|
|Litigation (Not Eligible)||2 (5.4)|
|Arbitration||1 ( 2.7)|
|Mixed or Aggregated Group||6 (16.2)|
|Business, Organization or NGO||13 (35.1)|
|Government||3 ( 8.1)|
|Not noted or other||9 (24.3)|
|Business, Organization or NGO||4 (10.8)|
|Not noted or other||8 (21.6)|
|Simplicity||4 ( 8.3)|
|Not noted||30 (62.5)|
In order to get around the difficulties of collecting data that may be confidential, difficult or impossible to obtain from all parties to a mediation or a comparison group, as well as to get information on perception-based issues (such as fairness) the majority of studies administered surveys or questionnaires to either the mediation group (87%) or the comparison group (84%).
The questionnaires administered to the mediation group were routinely longer and more detailed than those administered to the comparison group. Overall, this meta-analysis summarizes the responses of nearly 8,000 individuals’ comments upon their experience with a mediation program and over 2,000 individuals who commented upon their experience with a comparison group process.
|Variable||Number of Studies (k)||Average Number of Respondents per study (n)||Total Number of Respondents (N)|
|Surveys Administered to a Mediation Group||32||249||7,977|
|Surveys Administered to a Comparison Group||31||77||2,373|
There were enough programs and sites in the sample of studies to make many statistically significant statements about outcomes with high levels of confidence. When we state that we can be ‘confident’ about an outcome measure we mean that the studies we examined reported that mediation had a measurable impact. If it is noted that we cannot be ‘confident’ about an outcome measure we mean that the studies we examined were too few in number to provide us with good data or that results were mixed with some studies finding that mediation had a constructive impact and some finding a more detrimental impact. Moreover, it may not be merely variability or small sample that is the main issue, but that the impact of mediation on that outcome measure is actually very low, negligible or even possibly marginally negative. Unfortunately, we cannot distinguish between these possibilities in a meta-analysis. Therefore, the absence of ‘confidence’ could possibly indicate a more negative impact for the variable in question or it could be the result of a small and variable set of data. Hence, no firm conclusions should be inferred from findings in which we have low levels of ‘confidence’. Along these lines, there is a difficulty, in that the small sample sometimes contributed to very wide confidence interval ranges.
In our analysis we also examined ‘moderator’ variables, which influence the strength of a relationship between two other variables (Henriques 1999). The moderator analysis was undertaken to try to ascertain the characteristics that were most associated or disassociated with a particular outcome. The difficulty was that when examining the ‘moderator’ variables the number of programs or sites reporting this kind of information dropped even further. These small numbers resulted in low levels of confidence for much of the moderator analysis. As a result, there were very few outcome measures for which more than one moderator option could be discussed with any statistical confidence.
In only one instance was there a moderator variable that was negatively associated with an outcome measure. In this one case it could be compared to a positive association. There were many single instances, and some comparative instances, where positive associations could be drawn between moderators and an outcome measure. In the majority of moderator analyses there was only one statistically strong moderator having a positive association. In the vast remainder of the rest there were only vague differences in the degree of positive association of two moderators within one outcome measure. Therefore, the moderator analysis provided no major conclusions. This section presents detailed information about the fairly strong statements that can be made about aggregated measured outcomes and the meta-analysis of effect sizes of perception based measures, while listing how many moderators indicated a positive association with an outcome measure and a brief discussion of the few instances where more than one moderator can be examined for an outcome measure.
In all instances 95% confidence intervals have been calculated. A confidence interval (CI) is a statistic showing the range above and below the calculated mean where 19 times out of 20 the ‘true’ mean will actually appear. In many cases the confidence interval is detailed in the following analysis. The 95% confidence interval for which we are statistically confident in results are simply reported, while those 95% confidence intervals for which we are not statistically confident will be labelled with “n.s.” for ‘non-significant’. Those numbers for which we are not confident (i.e., n.s.) cross zero in their 95% confidence interval. They, therefore, may have the opposite association of that indicated by the mean number or they may be only the results of one study.
It should also be noted that by convention “k” is used to represent the number of study programs or sites which reported a sufficient amount of data to make a particular calculation. It does not reflect the number of studies which report any information about a certain issue, since congruent information about both the mediation group and a comparison group were required to make effect size calculations.
Please refer to Appendix 1 and 2 for details on the moderator analysis.
3.6.1 Time and Process Outcomes
A number of outcome measures were reported for time and process variables. They can be divided into two groups; those that are measured by mean improvements in caseload characteristics and those for which effect sizes can be calculated. For the mean improvements, these variables include the time saved by using the mediation program over a comparison group, the number of staff hours saved, the reduction in time from the initiation of a case to the resolution of a case, the reduction in the number of hearings or meetings, the reduction in the number of pre-trial conferences and the reduction in the number of motions. For the effect sizes, these variables include survey respondent agreement that the mediation program resulted in a net time savings over a comparison group, and agreement that a reduction in the number of motions had occurred.
We could calculate mean outcome measures for six variables assessing time and process outcomes, using between 6 and 27 programs or sites for the calculations. We could only be confident of the meta-analytic summary for the number of staff hours saved and the reduction of the total case length.
We could calculate mean outcome measures for staff hours saved in 13 different programs or sites. On average, cases that were processed through mediation demonstrated a savings of 61.1 hours in the amount of staff time an individual case took when compared to cases in a comparison group (plus or minus 27.51 hours, 19 times out of 20).
We could calculate mean outcome measures for reduction in case length in 27 different programs or sites. On average, cases that were processed through mediation were nearly 5 months shorter from initiation to resolution in mediation compared to cases in a comparison group (plus or minus 3.26 months, 19 times out of 20).
|Variable||Mean (k)||95% CI|
|Time Saved ||(n.s.)0.49 hr ( 8)||- .19 to + 1.17|
|Staff Hours Saved||61.1 hrs (13)||+ 33.59 to + 88.62|
|Case Length||4.99 mo (27)||+ 1.73 to + 8.26|
|Number of Hearing||(n.s.)0.21 ( 6)||- .05 to + 0.47|
|Pre-Trial Conferences||(n.s.)0.04 (11)||- .25 to + 0.33|
|Motions||(n.s.)0.67 (12)||- .18 to + 1.52|
We could calculate effect sizes for the perception of time savings in only 6 different programs or sites. On average, cases that were processed through mediation demonstrated a 31% improvement in perceptions of timeliness compared to cases in a comparison group (plus or minus 15%, 19 times out of 20). The differences in opinions between lawyers and litigants could not be accurately compared because the effect size estimate for time savings crossed zero in the case of litigant surveys (95% confidence interval between - 0.9 and + 0.33). However, based on four reporting studies, we can be confident(plus or minus 27%, 19 time out of 20) in stating that lawyers felt that mediation resulted in a time savings over a comparison group.
None of the examined studies asked for opinions regarding a reduction in motions.
|Variable||Mean ESE (k)||95% CI|
|Time Savings ||+ .31 ( 6)||+ .16 to + .46|
3.6.1.a Moderator Analysis of Time and Process Outcomes
There were a fair number of associations between a moderator option and improvements in certain time and process outcome measures; time savings (7), staff hours (23), case length (23), perceptions of time savings (17). The outcome measures for which there was no demonstrated improvement due to mediation predictably had very few moderator options that was associated with improvement in the outcome measure; number of hearings (1), number of pre-conference meetings (1), number of motions (0). Although these could be due to these options being more effective than other options within the moderator variable, without other options to compare them too it is also possible that they are artefacts of the small sample size.
The moderator analysis of time and process outcome measures only found 2 instances where different options within a moderator variable could be compared (i.e., for staff hours and case length). In the other 7 instances, where moderator variables could be compared the ranges of the confidence intervals overlapped, indicating that their impact was fairly similar.
The moderator analysis for staff hours found that there were two instances in the variable of freedom to select the mediator where the examined studies allowed us to confidently make a comparison. Where the selection of the mediator was voluntary (k=6) the associated reduction in staff hours was 97.8 (plus or minus 39.3 hours, 19 times out of 20), while where the selection of the mediator was mandatory (k=2) the associated reduction in staff hours was 50.0 hours. It could be that the voluntary selection of the mediation has more of an impact in reducing staff hours than mandatory mediation. However, when we see the small number of studies from which data on mandatory mediation was drawn (k=2) it does not appear that this conclusion can be supported very strongly.
The moderator analysis for case length found that it was possible to make statistically significant calculations regarding mean reductions in case length for comparison groups that were eligible for mediation (k=10) and litigation cases that were not eligible for mediation (k=2). For studies using a litigation eligible comparison group the associated reduction in case length was 4.49 months (plus or minus 3.67 months, 19 times out of 20), and for studies using a litigation ineligible comparison group the associated reduction in case length was 26.00 months. Thus, it could be that mediation programs do better than comparison groups that involve litigation of a type that is ineligible to go through mediation. Again, however, when we see the small number of studies from which data on mandatory mediation was drawn (k=2) it does not appear that this conclusion can be supported very strongly.
In the other cases where moderator variables could be compared the ranges of the confidence intervals overlapped, indicating that their impact was fairly similar, or were of little analytic value because one of the moderator variables was a catch-all category (e.g., ‘mixed’ or ‘other’).
3.6.2 Fairness and Satisfaction Outcomes
A large number of outcome measures were reported for perceptions of both fairness and satisfaction. Although the majority of outcome measures of this kind were subjective opinions, one was an objective measure of the case settlement rate. We could calculate effect sizes for all of the fairness and satisfaction outcome measures. Of the 6 outcome measures of fairness and satisfaction reported in the studies viewed only long-term satisfaction could not be analysed because only one study provided sufficient data on this outcome measure.
|Variable||Mean ESE (k)||95% CI|
|Cases Successfully Settled||+ 0.11 (20)||+ 0.05 to + 0.17|
|Perception of Fairness||+ 0.14 (17)||+ 0.04 to + 0.24|
|Satisfaction with the Outcome||+ 0.10 (15)||+ 0.04 to + 0.16|
|Satisfaction with the Process||+ 0.13 (11)||+ 0.03 to + 0.24|
|Perception of Compliance||+ 0.16 ( 5)||+ 0.06 to + 0.26|
We could calculate the effect size for the number of cases that were settled as a result of the mediation or comparison group process (i.e., the settlement rate). On average, the cases using a mediation group process demonstrated an 11% improvement in the rate at which cases were settled in contrast to a comparison group (plus or minus 6%, 19 times out of 20).
We can confidently assert the use of a mediation group process resulted in a 14% improvement in the perception that a process was fair or just when compared to some comparison group (plus or minus 10%, 19 times out of 20), in the 17 studies examined. Unfortunately, the difference in opinions regarding fairness between lawyers and litigants could not be compared because only one effect sized estimate for fairness could be calculated for lawyer respondents. Of the 17 effect sizes calculated using the litigant surveys the average improvement in the perceptions of fairness was 20% (plus or minus 7%, 19 times out of 20).
One type of outcome measure that was captured in many of the studies (k=15) was the percentage of the mediation group surveyed who agreed that they were satisfied with the outcome of the mediation. If it was unclear whether or not this kind of ‘personal satisfaction’ was being asked for instead of the ‘satisfaction with the dispute resolution process,’ the study was coded not with this outcome measure, but with the one examining satisfaction with the process. On average, cases that were processed through mediation demonstrated a 10% improvement in perceptions of personal satisfaction compared to cases in a comparison group (plus or minus 6%, 19 times out of 20). Unfortunately, we could not confidently compare lawyer and litigant impressions on this issue, since the statistical range of confidence for the lawyer surveys spanned both positive and negative numbers. However, of the 22 effect size estimates calculated for litigant surveys the demonstrated improvement in satisfaction with the outcome was 17% (plus or minus 6%, 19 times out of 20).
One of the dangers of asking subjective questions regarding satisfaction is that responses may be skewed by respondents whose are unable to distinguish between their level of satisfaction with the mediation program itself and the result (i.e., settlement) that they received from the mediation. This can clearly be the case when respondents are asked for their level of personal satisfaction, as they were done in the question reported in the previous paragraph. However, by structuring the question to examine the percentage of the mediation group surveyed agreeing that they were satisfied with the mediation program, one might be more likely to receive answers that are less coloured by individual bias. When outcome measures examining this kind of satisfaction are analysed (k=11), on average the cases that were processed through mediation demonstrated a 13% improvement in perceptions of satisfaction with the process compared to the comparison group (plus or minus 11%, 19 times out of 20). Again, the information provided did not allow any kind of confident comparison between the opinions of lawyers and litigants. However, in 18 surveys of litigants the demonstrated improvement in perceptions of satisfaction with the process was 20% (plus or minus 7%, 19 times out of 20).
Only one of the studies that was examined in this meta-analysis included sufficient data to calculate an effect size estimate for long-term satisfaction in some kind of secondary follow-up period. Thus, no aggregated summary of the effect of mediation on the long-term satisfaction could be made.
Some of the studies in the meta-analysis asked survey respondents if they thought that the agreement, settlement, disposition, resolution, etc. was being, or would be, complied with. Of the 5 studies reporting sufficient information, on average, cases that were processed through mediation demonstrated a 16% improvement in perceptions that the agreement would be complied with when compared to some comparison group (plus or minus 10%, 19 times out of 20). Again, no confident comparisons could be made between the opinions of lawyers and litigants. However, we can be confident in stating that the litigants surveyed demonstrated a 16% improvement in the perceptions that agreement would be complied with (plus or minus 5%, 19 times out of 20).
There were fewer surveys conducted of lawyers than of litigants, which made any comparisons between the groups difficult. The mean effect size for fairness and satisfaction outcome measures for lawyers always crossed zero, where the lowest confidence interval indicates a negative association and the highest confidence interval indicates a positive association, thus, providing no clear conclusions. Generally, we observe that the opinions of lawyers appeared to be more variable and include less positive assessments of the fairness and satisfaction outcome measures than those of litigants.
3.6.2.a Moderator Analysis of Fairness and Satisfaction Outcomes
There were a fair number of confident associations between a moderator option and improvements in certain fairness and satisfaction outcome measures; case settlement rate (21), perception of fairness (14), perception of satisfaction with outcome (22), perceptions of satisfaction with process (10), and perception of compliance (14).
In 12 of the 13 cases where moderator options for fairness and satisfaction outcome measures could be compared the ranges of the confidence intervals either overlapped, indicating that their impact was fairly similar, or were of little analytic value because one of the moderator variables was a catch-all category.
In one case the moderator analysis found that when there is full freedom to select the mediator (k=2) the studies reported results that demonstrates an average erosion of the case settlement rate of 6% (plus or minus 2%, 19 times out of 20), while when there is no freedom to select the mediator (k=13) their was a demonstrated average improvement in the settlement rate of 15% (plus or minus 8%, 19 times out of 20).
3.6.3 Cost Outcomes
A few outcome measures were reported for cost. They, in turn, can be divided into two groups; one that measured by mean improvements in caseload characteristics and those for which effect sizes can be calculated. One variable that was reported by a sufficient number of studies was the estimated or actual costs saved by using the mediation program over a comparison group, calculated as Canadian dollars. For the effect sizes, these variables include survey respondent agreement that the costs associated with mediation program were reasonable in contrast to a comparison group, and agreement that the mediation group process resulted in a net cost savings compared to a comparison group.
We could calculate effect sizes for perception of reasonable cost savings in only 5 different programs or sites. Unfortunately, the data reported in the examined studies cannot conclusively demonstrate an improvement in perceptions of reasonable cost compared to cases in a comparison group.
We could calculate effect sizes for perception of cost savings in 6 different programs or sites. On average, cases that were processed through mediation demonstrated a 29% improvement in perceptions of cost savings compared to cases in a comparison group (plus or minus 11%, 19 times out of 20). The differences in opinions between lawyers and litigants could not be accurately compared because the effect size estimate for cost savings crossed zero in the case of litigant surveys (95% confidence interval between - 0.41 and + 0.79). However, based on 5 studies surveying lawyers, we can be confident (plus or minus 13%, 19 time out of 20) in stating that lawyers felt that the mediation group program demonstrated a 30% improvement in cost savings compared to cases in a comparison group.
|Variable||Mean ESE (k)||95% CI|
|Perception of Reasonable Cost||(n.s.)+ 0.03 ( 5)||- 0.13 to + 0.18|
|Perception of Cost Savings||+ 0.29 ( 6)||+ 0.17 to + 0.40|
We could calculate a mean outcome measure for only one variable assessing cost outcomes, using 20 programs or sites for the calculation. As it was a commonly reported outcome measure in the studies we examined there was sufficient information to allow us to calculate the overall mean for many moderators.
As noted, we could calculate mean outcome measures for estimated or actual costs saved in 20 different programs or sites. On average, cases that were processed through mediation demonstrated a cost savings of $16,220 Canadian, per case, in the mediation group program compared to cases in a comparison group (plus or minus $8,402, 19 times out of 20).
|Variable||Mean (k)||95% CI|
|Costs Saved||C$ 16,220 (20)||+ 7,818 to + 24,623|
3.6.3.a Moderator Analysis of Cost Outcomes
There were a fair number of associations between a moderator option and improvements in certain cost outcome measures; perceptions of cost savings (21), and measured costs savings (27). For perceptions of reasonable cost there were none.
In all 5 cases where moderator options for cost outcome measures could be compared the ranges of the confidence intervals either overlapped, indicating that their impact was fairly similar, or were of little analytic value because one of the moderator variables was a catch-all category.
- Date modified: