Critical Review of Social Science Research on Parental Relocation Post-Separation/Divorce

2.0 Methodology

The purpose of this project has been to report on a comprehensive and critical review of Canadian and international social science research on relocation of families, particularly as it applies to families post-separation or divorce. To improve existing knowledge about relocation, this project included a Rapid Evidence Assessment (REA) of the social science evidence to retrieve studies that explored relocation within the context of separation and divorce. The REA approach was used to systematically detail the information retrieval process for the included and excluded studies, to assess the methodological quality of the relocation studies based on a standard assessment form, and to ensure transparency of the review process and results generated from the REA approach.

The review included published and unpublished Canadian and international (e.g., United States, England, Australia, and New Zealand) literature on relocation in the context of separation and divorce. Unpublished articles as well as conference presentations were identified through contact with key persons working in the area. The REA follows established guidelines for the inclusion of published reports (pre-determined inclusion criteria), standard critical appraisal of the evidence and data synthesis to ensure the information retrieval process is explicit and that the criteria for inclusion and exclusion of studies are transparent.

2.1 The search for relevant studies

When searching for published studies in electronic databases the following relevant electronic databases were included: Medline; Sociological abstracts; ASSIA; ERIC; Digital Dissertations @ Scholars Portal; Social Services Abstracts; Social Sciences Citation Index; Family Studies Abstracts; CINHAL; EMBASE; All EBM Reviews - Cochrane DSR, ACP Journal Club, DARE, CCTR, CMR, HTA, and NHSEED; and ISI Web of Knowledge.

Grey literature articles (i.e. unpublished manuscripts, conference proceedings, topical bibliographies, and curriculum vitae’s lists) were searched by: Internet search engines (Google, Yahoo, and Altavista), government websites, and organizational websites (CECW, CWLA, etc.). Search term combinations inputted in each database included the following: (exp relocation; exp mobility; geographic; geographic; geographic; residential; residential ) AND (exp divorce/;;; relationship termination; marital separation; divorced persons).

2.2 Critical appraisal of the evidence

To be included in the analysis, included studies needed to have some semblance of a qualitative or quantitative research design, an indication of how cases were selected for the study, information about the data gathering procedures and measures employed, and information about the data analysis methods, along with the findings. This means that numerous clinical and opinion, court file analyses and review articles on the subject of relocation were not included as they do not provide this important information about research design that enables one to evaluate their findingsFootnote 3. Furthermore, many of these review articles mix findings related to relocation of divorced populations with other indirect findings about children and families in the general population (e.g. overview of local moves in non-divorced populations) and parent-child relationships generally (see for example, Austin, 2012).

In assessing the credibility and precision of the current scientific evidence, it is important to recognize that not all research designs are equal in minimizing biases and controlling for risk of error in the results (Saini, Johnston, Fidler & Bala, 2012). Some research methods provide better evidence than other methods when seeking answers to specific questions. Saini et al., (2012) note that qualitative interviews are preferred for an in-depth exploration of participants’ experiences and are valued for their hypothesis-generating capacity, but they are not well suited for making inferences beyond the study sample. Qualitative studies often include small sample sizes, in-depth interviewing techniques, and consider the local contextual factors of the sample, making it difficult and perhaps misleading to make generalizations beyond the sample participants. To make more sound inferences beyond the sample, it is best to use quantitative surveys using random sampling techniques from a known population. Moreover, when randomized control comparison groups are used within quantitative surveys, studies are better able to isolate any associational relationships found in the target sample that are different from the comparison group.

Methodological rating of the quality of the studies were based on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system(Atkins, Eccles, Flottorp, Guyatt , Henry Hill, et al., 2004). In rating the quality of evidence across study designs, it is important to consider the unique methodological considerations for each of the methods used, as there is no “one-size-fits-all” approach for assessing quality (Saini et al., 2012). The quality assessment tool used in this report, therefore, has been adapted from conventional quality appraisal tools to provide sufficient flexibility in rating studies across designs. The author acknowledges, however, that this quality assessment tool favours quantitative designs because of its ability to produce empirical generalizations beyond the samples included.

Determination of quality was determined by the GRADE over eight specific dimensions:

  1. Did the study use a random selection from the population parameters that would allow one to generalize the results of the study widely to other similar populations?
  2. Did the study use a comparison or control group that helps to verify the hypothesized preconditions or presence of the effect (or conduct systematic intra-group comparisons)?
  3. Did the study use standard measures (those consistently applied within the study) or standardized measures with reported psychometric properties (those consistently applied across studies) for the dependent (DV) and independent variables (IV)?
  4. Are data gathered from multiple sources of informants (versus a single source) so that different perspectives of relevant observers are considered (e.g. mothers, fathers, child, clinician, etc.)?
  5. Did the study systematically control for extraneous variables that may have influenced the magnitude of the effect (e.g. influence of siblings, age, gender), and/or alternative explanatory factors for the effect (e.g. inter-parental conflict that might explain long term outcomes or problematic/abusive parenting that might explain the child’s attitudes to the relocation)?
  6. Did the study design establish a temporal order between the dependent and independent variables in order to test for direction of effects or causality?
  7. Were the selection and exclusion criteria, response rates and subject attrition explicitly defined and explained so the kind of sample the findings pertain to is clear?
  8. Is there sufficient sample power (as determined by sample size, and magnitude of expected effects for independent and control variables) in order to be able to detect robust, statistically significant and clinically important findings?

2.3 Scoring the studies

Scoring of the GRADE was based on calculated the scores across the eight dimensions of quality and dividing by two, which resulted in a total score of quality according to the GRADE criteria of quality. The reason for dividing the eight dimensions by two is to create four categories of quality, from very low quality to high quality.

Very Low Quality
(scores 2 or less). Any estimate of effect is very uncertain.
Low Quality
(scores 3-4). Further research is very likely to have an important impact on confidence in the estimate of effect and is likely to change the estimate.
Moderate Quality
(scores 5-6). Further research is likely to have an important impact on confidence in the estimate of effect and may change the estimate.
High Quality
(scores 7-8). Further research is very unlikely to change confidence in the estimate of effect.

The GRADE has been used previously to assess the quality of empirical research within the separation and divorce field. Saini, et al., (2012), for example, used the GRADE to assess the quality of evidence regarding 39 empirical studies on alienation. Assessing the quality of the methodological strengths and limitations of the empirical evidence used to support legal positions in family law matters provides the reader with critical information about the potential strengths and limits of applying the evidence to client-based decisions.

Date modified: