systematic literature review

systematic literature review

The Importance of Systematic Literature Reviews

1. Introduction

Recent years have seen a growing interest in systematic reviews as an important research methodology in its own right. A number of subject-specific journals have been launched, and a firm foundation of research has been developed across a wide range of disciplines. The Cochrane Collaboration, established only a little over a decade ago, has become a well-recognized international organization that has played a leading role in promoting and facilitating systematic reviews. The application of systematic reviews extends beyond reviews of clinical interventions, and we are beginning to see considerable development of systematic reviews of other types of research, such as public health interventions, and social and psychological studies. This growing movement has generated much discussion and debate about the relative merits of systematic reviews over other approaches to reviewing research evidence. At the same time, there has been persistence of misunderstanding about, and prejudice against this valuable approach to research synthesis.

2. Benefits of Conducting a Systematic Literature Review

Systematic reviews, a powerful way of minimizing bias, can be applied to practically any area of research. Systematic reviews offer a number of advantages to health care: we often hear that more research is needed. Systematic reviews can establish what is already known and can help to identify where further research is necessary. By summarizing the existing literature on a topic, systematic reviews can help to make health care decisions more effective. For example, there may be several studies conducted in different countries on the same issue. These could provide a consistent message on the effectiveness of an intervention, but because they are published in different places, some could remain unknown to those who could benefit from the study. Other research might contradict each other. By bringing together all the evidence on a specific topic, systematic reviews can act as a final piece to the jigsaw and help to translate research findings into practice. With many individual studies taking place every year, reviews are a more time-effective way of informing policy and practice in health care. Systematic reviews are also valuable in predicting future trends. By highlighting an area in which there is little research, or a topic on which several inconclusive or contradictory studies have taken place, they can help to shape the research agenda. This is important in the Cochrane Collaboration, as priorities need to be set for the topics of Cochrane protocols or new Cochrane reviews. This is also pertinent to policy makers, who can use reviews in making decisions and setting agendas for the future.

3. Steps in Conducting a Systematic Literature Review

The next step in conducting the review is to locate the literature. This will involve searches of electronic databases, web searches, reviewing citations from other sources, and hand searching. Hand searching involves looking through specific journals or conference proceedings which are not currently indexed in electronic databases. It is generally only practical to locate the most recent literature for the review. The time it would take to locate all literature throughout history would be overwhelming. A decision must be made regarding how recent a paper must be to be included in the review. Above five years old is generally considered a non-recent source, but this can vary significantly between topic areas. Any literature obtained must then have its details recorded and be organized to facilitate easy access.

Developing a protocol specifying the methods to be used in conducting a literature review is a critical step. Protocol development involves deciding on the review question, inclusion/exclusion criteria for the literature to be reviewed, and the requirements for article acceptability. Such decisions will be based on the epistemological stance of the reviewers. Even if this is not explicitly stated, those with a post-positivist view will specify clear criteria to maximize objectivity, while those with another epistemological view may specify criteria to ensure the literature to be reviewed is representative of their beliefs about the phenomenon in question.

4. Challenges and Limitations of Systematic Literature Reviews

The process is dependent on the quality of the literature (Adams et al., 2009) and without access to good quality literature, the value of the review is limited. The grey literature is always difficult to access, although attempts have been made to make this easier in some areas such as the Cochrane Collaboration (Booth, 2006). This can have an impact on the overall findings of the review. For example, a recent review on the effectiveness of exercise in fibromyalgia found that studies published in languages other than English were of a significantly smaller sample size (9.5 vs 52.1, p=0.008) and resulted in significantly less positive effects compared with those published in English (Busch et al., 2008). Although other reasons for this observation cannot be ruled out, it suggests that limiting the search to English literature may result in missing data in a systematic review. Limitations with access to literature are likely to be imposed further in the future by the restriction of access to open access articles or those funded by the National Institute of Health due to a decision made by the US government and implemented by the NIH to require researchers to make articles arising from NIH funds available through PubMed Central within 12 months of publication (National Library of Health, 2005). This has been seen as a backwards step for public health (Adams et al., 2009).

5. Conclusion

SRs are an ever more popular and powerful tool for studying research questions. SR methods are also becoming increasingly used in evidence-based software engineering (EBSE) in order to provide a rigorous and relevant synthesis of primary studies in a specific domain. SRs originated in the medical field and quickly spread to other scientific fields. They aim to comprehensively identify and synthesize all relevant research on a particular question in a way that is unbiased and highly transparent. Therefore, they are an attractive method for minimizing the risks of research waste caused by study replication or performing near-identical studies, as well as for ensuring that research findings are methodologically and empirically grounded. SRs involve defining a clear research question, explicit and highly systematic methods for identifying and including studies, and study quality assessment. Systematic mapping is a related technique that is used to catalogue the amount and type of research available in a certain area. Although it has some similarity to an SR, the lack of a synthesis phase means that it is not actually a study. In this paper, the author stresses the need for more primary research in software engineering to create a stronger knowledge base for the field. While not arguing against this, we believe that there is a place for more secondary research in the form of SRs and mapping studies in order to produce greater understanding of the current breadth and depth of the research topics, actively reduce waste, and identify in which areas primary research would be most beneficial.