systematic literature review methodology

systematic literature review methodology

The Importance of Systematic Literature Review Methodology

1. Introduction

This makes it an important task to evaluate the methods implemented in the study being appraised. The Cochrane collaboration provides 2 highly regarded tools for evaluation in this area. The first being the ‘Cochrane risk of Bias Assessment Tool’ which is designed to evaluate the risk of bias present in a study and the ‘Cochrane Effective Practice and Organization of Care Review Group Data Collection Checklist.’ Guidelines exist for evaluating a wide variety of study types. An example of this is the ‘Critical Appraisal Skills Programme’ for randomized controlled trials and cohort studies. The results of these methodological reviews are then considered in the context of the primary research and its implications on practice.

The rationale behind methodological quality systematic reviews has been previously outlined. In addition to this, in order to ensure an unequivocal improvement in healthcare, it is essential that clinical decisions are based on the most reliable evidence available. Therefore, systematic reviews of healthcare interventions should include only those studies which have evaluated the effects of an intervention in a systematic and unbiased manner. This is because these are the studies which will yield the most reliable evidence.

There is currently a massive amount of literature encompassing systematic literature reviews, which are a substantial proportion of the medical, scientific, and social science research initiative. The purpose of this article is to highlight the need for better methodological quality of primary and secondary reports of systematic reviews. There are several reasons why high-quality methodological reviews are important. The intention herein is to outline the most essential types of systematic reviews and meta-analyses concerning methodological reviews, examining the strengths and weaknesses of individual studies. This can be achieved in a number of ways. The authors of this paper have written extensive guidance for both those proposing to do.

2. Benefits of Systematic Literature Review Methodology

A systematic review offers a solution to the problems of both narrative and vote-counting approaches. By adhering to a careful method of planning, execution, and interpretation of available research, a systematic review aims to minimize bias and provide reliable findings. It does not force an a priori framework onto the research, as with some interpretive forms, but rather seeks to identify all relevant research and appraise its quality and relevance through a thorough and explicit method of evidence synthesis. By this means, systematic review is increasingly seen as a key research methodology, being rigorously applied in fields other than clinical medicine and health sciences. Consumers of research often find systematic reviews to be the most reliable and useful way of appraising the state of knowledge on a topic. A Cochrane group established in 1992 has been particularly effective in this arena. Cochrane reviews are systematic reviews of healthcare interventions, and are recognized as representing a gold standard in evidence-based health care (Higgins and Green, 2006).

Several alternative approaches for reviewing the literature on leadership exist, including narrative reviews and vote counting. Systematic review methodology can be traced back to the movement to increase the validity of clinical practice and the development of evidence-based medicine (Smith, 1999). Narrative reviews are valuable for integrating and interpreting the state of a field, and can even take into account evidence outside of established research designs, but they are particularly at risk for bias and error. Vote-counting, whether through a straightforward tally or a more complex synthesis of statistical outcomes, has been the subject of much criticism for its lack of interpretation and limited applicability to actual practice.

A systemic literature review of actual leadership behaviors.

3. Steps involved in Systematic Literature Review Methodology

Step 4: Evaluating the reports found – In most cases, an initial classification of reports can be made immediately after the full text is obtained, and sometimes it will be evident that some of the included reports do not usefully address the review topic. For specific study designs or topic areas, there may be an evidence hierarchy with respect to the relevance and validity of the reports, and it may be decided to include only the reports that are above a certain threshold. This anticipated assessment process is defined by criteria, which should be documented in the context of the review. All the information and data that are detailed in the following steps are relevant to the studies included and should be extracted and documented on forms specifically designed for this purpose. At the end of data extraction, it is essential to assess the quality of the data and documentation (of the review process, and how the decisions were made, and the source of the information with date). This requires periodic checks and review by all reviewers in the review team to ensure that they are doing a good job.

Step 3: Selection – This involves two specific stages. The first stage is a preliminary screening of the titles and abstracts to exclude material that is obviously irrelevant. At least two reviewers should work independently on all these stages. This is followed by obtaining the full text of the remaining reports and an assessment of these against the pre-defined eligibility criteria. In both stages, a record is kept of the number of reports excluded, and reasons for exclusions are provided. Again, decisions made during the selection process are documented. This will allow the review to be updated at a later time. This is the minimum requirement for a systematic review. The objective should be a comprehensive and exhaustive review. At the end of this stage, it is also useful to compare the characteristics and results of the included reports with the intended scope and objectives as stated in the review protocol. This will indicate any unplanned deviations. This is not necessarily bad but should be documented and will require a judgment whether to modify the protocol.

Step 2: Locating and Collecting the Information – In this stage, the reviewer needs to consider where he is likely to find the studies that have information which he seeks towards answering his research question posed in step 1. This involves a detailed documentation of which information sources were consulted, such as the name of which databases and library catalogues with the dates of coverage and the search features used, and also includes the list of the journals consulted. This is done to minimize publication and selection bias. All decisions made at this stage are also documented to provide a record of the review process and for use in an update of the review. An EndNote library is set up to include all references at a single location.

Step 1: Defining the research question – This is one of the most important steps of all. It is essential that the reviewers have a strong understanding of the area under study and the objectives of the research. It often happens that the first search produces a large number of hits, and the automated selection of abstracts, or abstracts and titles, for the further retrieval step may have to be iterated with a different search by using different terms. At the end, the reviewers may realize that they were actually looking for another concept as the search evolves with increased understanding. This will require going back to the search of this previous concept.

The number of steps involved in SLR has been a point of debate, but several researchers have considered a 6-step approach by referring to the works of several authors. We shall be considering a 6-step approach as suggested by Wright et al. (2007) and also considering the viewpoints of other authors.

4. Challenges in Implementing Systematic Literature Review Methodology

Managing the complexity and volume of information retrieved in systematic reviews can also be challenging, and problems in doing so can lead to a review’s failure. A balance of end-user need with the feasibility of the review must be maintained. Therefore, a decision needs to be made on how much evidence is enough. The review should not continue beyond a point where the marginal benefit of answering the review question with further evidence is exceeded by the marginal cost of conducting the review. Endeavors to be too exhaustive can lead to wasted resources with no added value to the end user, as reviews are sometimes abandoned with no dissemination of results. Steps can be taken at the selection phase for inclusion of studies to manage the volume of information. For example, limiting the review to English language studies, studies of certain quality, or from a certain location. An appropriate and regular contact with the end user requiring constant refinement of the research question and objectives will also help to make sure that the review stays focused and does not digress into unrelated areas of investigation.

Optimal systematic reviews require top quality from the outset. This includes a well-thought-out protocol, seeking advice from information specialists in constructing a comprehensive search strategy, conducting a wide-ranging and unbiased search, reaching the right decisions on what to include and exclude, and a thorough assessment of the retrieved information. The need for quality at each stage in the review process can be challenging in that it may require a reviewer to rethink and repeat a process that can be time-consuming. For example, employing various search terms and combinations to access the most relevant information. High-quality reviews can also be threatened by publication bias. Publication bias occurs when the results from research studies influence the decision of whether to publish or not, and if so, what to publish. Findings that are less clear or inconclusive are often not published. Publication bias can affect the decisions of researchers conducting systematic reviews to include or exclude certain studies. However, those studies may contain the most unbiased and accurate data available from the research. This makes a thorough search and unbiased inclusion criteria essential.

For all its superiority as a technique for bringing evidence to the forefront in any given field of enquiry, there are numerous challenges in implementing systematic literature review methodology. Specifically, the challenges involve extensive resources required, time, quality, and managing the review process. The extensive resource required by systematic reviews can make them a luxury. This is especially true of review teams whose members do not have access to a rich and wide-ranging database collection. The review projects in such instances have been known to run out of steam due to lack of access to resources. Creating meta-analyses and undertaking extensive searches in order to minimize bias are particular consumers of resources, though it should not be forgotten that evidence and conclusions drawn by systematic reviews are potentially of great value to resource-poor decision makers. In such instances, the reviews cannot be viewed as a luxury. Systematic reviews can require large amounts of time. High-quality reviews may take a year or more to complete, particularly if researchers are seeking to inform complex or controversial debates. While the necessity of substantial time inputs is not necessarily a challenge because the review may be tackling an issue where resources are best not spared, it is a factor that can inhibit enthusiasm for a review. Indeed, sometimes there will be no point where the reviewer can be sure that this is an issue worth the time it will take to illuminate it.

5. Conclusion

Slr as a methodological tool is of utmost importance. It is widely used in healthcare technology and informatics due to its thorough and unbiased approach. There is a growing body of work discussing the use of SLR in this field, together with a variety of tools, but little that specifically details the methodological requirements unique to this type of review. The aim is that this paper will provide a useful reference for those wanting to use SLR in healthcare technology and informatics research, by providing a clear methodological framework together with elaboration and justification of the steps involved. This should reduce the potential for ad-hoc and incomplete review and encourage the replication of high quality reviews in the future. With the increasing interest in this type of review and a lack of consensus regarding methodology, it is hoped that this paper will be timely and a valuable resource for researchers in this field.