Systematic review article and meta-analyses constitute a group of Level 1 and 2 studies and are considered at the highest level of evidence-based medicine (EBM) 1. Whereas unsystematic, narrative review articles are marred by author bias, as it is more probable to embrace only research chosen by the authors, thereby introducing bias. We believe that since systematic review articles are becoming more easily understood and accepted in medical literature, they will eventually be the format accepted for the reviews and possible publication in indexed journals. This blog will allow our readers to comfortably comprehend these reviews and feel at ease in formatting their own systematic review article in the future.
The ability to carry out an enlightening systematic review of the available literature, and its inherent limitations, quality and potential are an essential skill for any researcher to develop. Not only does it help answer the hypothetical question, it also enables valuable information and provides guidance about the planning and recommendation of the value of any innovative research.
The process of writing systematic review articles takes place in order to put multiple studies or researches into an organized form so that useful information can be extracted from. Writing a good systematic review directly determines the entire basis of how good a meta-analysis of any research program is. Therefore, more competently the conducted systematic reviews are, more useful conclusions can be drawn from the relative studies. A systematic literature review is a crucial step of evidence-based medicine (EBM). It does not only aim to organize studies and make a useful interpretation based on them but also helps in an appropriate assessment of the quality of different studies. Thus making your research process very high yield and enables targeted questions to be answered through it.
A systematic literature review can be defined as an endeavor to “identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question” (Cochrane definition, 20132).
The aims of a systematic review can be varied and include:
- Elucidating the relative strengths and drawbacks of the available literature on the question,
- Summarizing an enormous expanse of literature,
- Resolving literature disagreements,
- Appraising the necessity for a large clinical trial,
- Circumventing the need for an unwarranted trial,
- Increasing the statistical power of smaller studies,
- Improving the precision or identify a smaller treatment effect, and
- Improving the generalizability of treatment outcomes.
Types of Systematic Review Article:
The types of systematic reviews for any literature can be defined more as steps of drawing a conclusion on evidence collected from the literature. At every step, the information becomes more refined and properly defined. The three types of systematic reviews are:
- Qualitative reviews: This type of systematic review is based on summarizing the studies that have been conducted on a subject.
- Quantitative reviews: Quantitative reviews make use of combining the statistical data or conclusions of different studies together. The aim is to organize two or more than two multiple studies for drawing useful conclusions.
- Meta-analysis: Meta-analysis is the process of bringing together the data collected on the basis of systematic reviews. This is more of a proceeding step that follows systematic reviews.
Steps for Conducting a Systematic Review:
A sound systematic review needs a few protocols to be followed. Following these protocols and carefully taking all the essential steps assures that the systematic review conducted is high-yield. Systematic reviews are listed as one of the top forms of evidence-based medicine (EBM). The essence of a good EBM lies in its forming elements like meta-analysis and systematic reviews being efficient and decisive.
We have briefed the pivotal steps that are to be respected while conducting a systematic review.
1) Formulation of a Research Question
2) Study Selection Criteria
3) Developing a Research Protocol
4) Data Extraction
5) Data Analysis
6) Interpretation of Results
1) Formulation of a Research Question:
A hypothesis could be defined as a projected rationalization for an observation. It could either be investigated ( the scientific hypothesis) or it might be carried forward (working hypothesis).
- You need a primary research question in order conduct a sound systematic review. Your systematic review has to revolve around this question. An efficient research question typically comprises of;
- Patient groups studied;
- Tests, or exposure for the population;
- Alternative intervention or Control; and
- Results of the interventions.
This first step is crucial because it gives your project a clear-cut direction, which can be followed to reach the targeted goal. A number of useful questions can be addressed which elaborate any condition in a better way. For instance, the etiology of the disease, the results of different diagnostic tools, the occurrence of disease, and certain other relevant things can be addressed in this. Developing a primary research question thus gives a firm basis to the conduction of your systematic review article.
Please keep in mind that a vague hypothesis/question would most likely to lead you to a vague conclusion/answer.
Good systematic review questions could address either of the following:
- Phenomena associated with disease or interventions
- Disease or its related condition frequency
- Diagnostic precision
- Disease etiology and/or its risk factors
- Prognosis, and
- Efficacy of the intervention studied
2) Study Selection Criteria:
It is important to identify the relevant studies that can be specifically helpful for your research. Some criteria have to be set which determines the inclusion or exclusion of a certain study in your program. You are advised to carry out extensive research on different levels of studies that are available regardless of the language limitations. A few researchers may find it exhausting and strenuous to collect data from different languages but integrating the studies from different languages can largely extend the scope of your study.
After the extensive mass of literature has been collected, it is necessary to filter them on the basis of your study selection criteria. Your criteria should aim to select those studies that can be highly efficient for your subject and exclude those studies, which are less relevant.
3) Developing a Research Protocol:
Developing a research protocol is important because it helps to develop an outline of your research goals. It is a precise yet comprehensive explanation of your literature goals and is thus important in organizing the data and information. Nowadays journals require you to submit the protocol along with your manuscript. Authors of systematic reviews are expected to make use of the either EQUATOR (http://www.equator-network.org/) or PRISMA statement or similar other guidelines to format their protocol. The PRISMA Statement consists of a checklist (27-items) and is accompanied by a flow diagram. It is a guide for authors about how to develop a protocol for the review and what needs to be included while writing the systematic review.
4) Data Extraction:
Data extraction is important because helps in reaching to the useful conclusions. The common method of data extraction is using an electronic or paper standardized form. This is by far the most efficient method of data extraction and is used worldwide by a large number of people. A standardized data extraction form comprises of certain essentials like the reference of the study. It also explains the objective of the study and the type of population chosen for the study process. Other factors like the demographic analysis; intervention control, and outcomes are also discussed in the standardized data extraction forms.
5) Data Analysis:
The step of data analysis further polishes the information that you are collecting for your systematic review. Other useful techniques for serving the purpose may consist of quality appraisal and study quality assessment. The most efficient method for data analysis is to present your study evaluation in a tabulated form. Tabulation of the study helps in clearly understanding the results obtained from the data. All the interventions, the control and study groups, the outcomes and other important information should be presented in the data analysis.
6) Interpretation of Results:
The correct interpretation of results determines the quality of your systematic review and it needs to be carried out with caution. Poor interpretation of results always signals to the low quality of the study and can be characterized by poorly defined conclusions.
Once you have collected sufficient quantitative data, it is advised to put it through an extensive meta-analysis, using statistical methods. If you are not confident, please do consult a statistician before the review begins. As a starter, a simple tip is provided. If you are conducting a study to assess the collective efficacy of a drug across all studies in the literature, an effective method of merging different measures of outcomes is to use odds ratios calculated from each outcome with 95% confidence intervals. Follow this with analysis of statistical significance to estimate the degree of effect and heterogeneity between all the studies. Ranked by effect size, the data could then be plotted into a ‘forest-plot’.
Common limitations with systematic review article could include:
- Important studies are neglected due to an inadequate literature search.
- Too many biases present.
- Vague hypothesis or research question.
- Ambiguous review methodology.
- The findings of the review lead to inconclusive inference.
A systematic review article benefits healthcare professionals to stay conversant with the precipitously changing medical literature. Hence, a good systematic review article should be able to defend that it conducted an all-encompassing systematic search, identified, selected, evaluated, and amalgamated research evidence pertinent to the hypothesis/question, using a methodology that was unambiguous, reproducible, and thereby incorporated the least bias.
By being mindful of our easy-to-follow step-by-step guide for writing a systematic review, you are sure to conduct the process in an efficient manner. This holds true for those who already have experience in this regard as well as for those who do not have a formal training of the process. Any researcher who wants to succeed in presenting their research as a competitive distribution needs to carry out the statistical analysis in a proficient way. Once you realize the importance of carrying out our research in a systemized way, you can conduct a quality systematic review by following our aforementioned points.
- Wright RW et al. Integrating Evidence-based Medicine into Clinical Practice. J Bone Joint Surg Am. 2007;89: 199–205.
- The Cochrane Library; http://www.thecochranelibrary.com/view/0/index.html
- EQUATOR Network. Available at: www.equator-network.org.
- The Cochrane Collaboration. The reliable source of evidence in healthcare. Available at: www.cochrane.org.
- CEBM Centre for Evidence-based Medicine. Available at: www.cbem.net
- The Campbell Collaboration. Available at: www.campbellcollaboration.org.
To analyze what has already been analyzed is Meta Analysis
Meta-analysis is a valuable process to condense data incorporating many studies but necessitates meticulous consideration, organization, and execution. Meta-analysis involves investing substantial time and energy in development and accomplishment. Thereby, it should not be considered as a nippy way to collect and analyze studies with a sole aim of making a quick publication. Hasselblad wrote an excellent article, with lots of examples and formulae that would be of great value for any researcher who wishes to initiate a meta-analysis1.
After the process of accomplishing systematic review has been completed in any study, the process of conducting meta-analysis can take place subsequently. To put in simple words, meta-analysis is the process coalescing the data collected on the basis of systematic reviews. Combining this data together helps in drawing useful conclusions in any study. However, not all systematic reviews necessarily lead to a good meta-analysis. There are some crucial points that govern the efficient conduction of a meta-analysis that is up to the mark. Being one of the most important statistical approaches, meta-analysis has a direct impact on the outcomes of any research. Therefore, before starting the process, it is necessary to clearly understand the factors that will make your meta-analysis high yield. The researchers who understand the basics of conducting a good meta-analysis draw far more useful conclusions than those who fail at this crucial step of their research.
Aims to be considered when you conduct a Meta-analysis:
- Construct statistical importance concerning studies with diverging results
- Develop an accurate appraisal of the effect magnitude
- Provide extensive complex analysis of harms, safety data, and benefits
- Scrutinize subgroups with individual numbers that are not statistically significant
Advantages and Disadvantages of Meta Analysis
|Superior statistical power||Heterogeneity of study populations|
|Confirmatory data analysis||Necessitates advanced statistical methods|
|Better capability to generalize to the general population affected||Challenging and time-consuming effort required to recognize apt studies|
|Measured as an evidence-based source||All studies do not specify acceptable data for inclusion and analysis|
In this blog, we explain how a meta-analysis is similar to a traditional study, involving a written protocol with design elements that are analogous to a systematic review. We have highlighted some pivotal points you might need to comprehend before conducting any meta-analysis. We will concentrate on the imperative issues fundamental for a meta-analysis: articulating the hypothesis/study question, selection of research studies, accumulating and appraising evidence from these studies, and extracting fruitful results.
Step-By-Step Guide to conduct your Meta Analysis
- Step 1: Identify studies and Employ Inclusion/Exclusion criteria to Titles and Abstracts
- Step 2: Exclude Studies that evidently meet the Exclusion Criteria predetermined by you
- Step 3: Download and save the full text of the remaining research studies/articles
- Step 4: Assess the studies to see if they concur to your Inclusion and Exclusion criteria
- Step 5: Only embrace studies that fulfill all your Inclusion criteria and No Exclusion Criteria
- Step 6: Eliminate studies from Meta Analysis with logical reasoning
- Step 7: Accept the remaining studies to initiate your Meta Analysis
IDENTIFICATION (Literature Search for a Meta Analysis):
This is the first and the most important characteristic of any good and solid meta-analysis. You have to make sure that your literature search is thorough, extensive, comprehensive, and well systemized. Important sources of information for a meta-analysis include:
- The Cochrane Collaboration Controlled Trials Register
- MEDLINE/EMBASE (The European version of MEDLINE is called EMBASE, and is a Dutch/English collaboration)
- CancerLit, AIDSLine, and ToxLine
- Index Medicus
You should comprehensively cover all the studies that are relevant to your subject or the topic. Look for all the published as well as unpublished research work that relates to your topic. There is a term used ‘fugitive literature’. This literature refers to the research work that could not be published but can be of good help to the researchers. It is therefore strictly advised to make the best use of dissertations, non-indexed studies, and unpublished studies to make your analysis even better.
Collaboration with Colleagues:
We would suggest that if you plan to do a meta-analysis of your own, it might be pertinent that you consider enlisting the help of an expert such as the medical librarian. It is also crucial to build good collaboration with other researchers that are involved in the programs similar to yours. This is important because they can provide you with useful information that can help you in establishing your meta-analysis. You can also ask for the workers in these relevant projects to join you in your program. You should also look for your undergraduate colleagues who are looking for course credits. They may demand course credits from you and in return can provide you with useful literature search. This does not only save time but also helps to extend the array of your research.
References and Citations:
Look for the maximum number of references and citations that may help you understand the whole process in detail. Once you have got a good grasp on knowing how the entire process is carried out, you are sure to conduct your analysis in a sound manner.
Set Homogenous Standards:
Any good meta-analysis needs a homogenous standard to which the results obtained from the data can be standardized. This facilitates the process of comparing the different results and therefore saves a lot of time. This also helps in understanding the relevant risks that help in establishing a good meta-analysis.
SELECTION (Eligibility Criteria):
You would agree that all research are not created equal, hence, the use of substantiation to guide the clinical protocols needs to be conscious of the supporting research that propagates diverse interventions. Once you have collected a large set of studies that can be helpful for analysis, it is important to exclude the less relevant content.
There are various achievable inclusion/ eligibility criteria:
- Did the study comprise of enough information for analysis?
- Did the research ensure the validity of the study design?
- Did the study meet standards for a minimum sample size?
- What was the drug dosage used in the study?
- Was language of the article comprehensible?
- The patient age, sex and other credentialing?
- Did the research define the study setting?
This facilitates in drawing conclusions and results that are to the point and more efficient. Assembling this set of requirements that a study needs to meet in order to be a part of your research is known as the eligibility criteria.
The first thing you need to set eligibility criteria is to see if the study contains enough useful information. Then you have to see if the study design is optimized to your research. Other useful criteria can be the language of the study, the time at which it was conducted and how this impacts your meta-analysis, age group of the patients on which the study was conducted, the clinical setting in which the study was conducted for instance OPD, inpatient, or emergency room. All these factors can directly influence your meta-analysis and hence it is necessary.
The role of the data abstraction phase pertains to the assessment of the study quality. The results of the quality assessment should describe the breakdown and understanding of the results.
Once you have a suitable collection of studies that meet your eligibility criteria, you have had to abstract the applicable data from each study accumulated. The potential error in data abstraction could be because of:
- Typographical or copyediting errors
- Misinterpreted Tables and Charts
- Erroneous data entry or abstraction process
Precise Coding Forms:
Coding forms are the basis of any good meta-analysis. Make sure you establish such coding forms that are very precise and provide only relevant statistics. A systematic coding form is characterized by containing only such information that is guaranteed to draw high yield results and conclusions. In order to keep these forms precise and easily comprehensible, you are advised to keep only those variables that you have to test in your program.
Keep Systematic Records:
It is really crucial to keep well-systemized records of everything that you do in your meta-analysis. From keeping a track of your literature search to drawing the final conclusions, every little information is important. This is also important when you explain the method section of your paper at the end of your research.
How to Avoid Errors
To minimize errors a good meta-analysis should follow simple steps like:
- Use two or more independent reviewers or have consensus meetings to decide about any conflicts
- Try to educate reviewers by making them practice analysis of the research by reading various articles so that every reviewer standardizes to a common goal
- Always indulge frequently in comparison of abstracts and texts to unearth discrepancies in the studies
- It is desirable to utilize a standard form/database that restrains studies to the projected range
Meta Analysis is closely marred by many controversies regarding the analysis of study data. Before we dwell into all those, let’s define some critical terms:
Homogeneity indicates how analogous the results of different studies were to one another on a fair comparison.
Heterogeneity refers to how different the results are between studies. A cluster of studies having dissimilar results is said to be heterogeneous. Simply put, it is the opposite of homogeneity.
Fixed effects models deliberate upon only the within-study variability. It is assumed that the studies utilize identical methods, patients, and measurements; hence producing identical results; those variations are because of within-study variation. The researcher by exploiting a fixed effects model can answer the question: “Was the treatment able to construct a benefit on average in the studies?” If the studies are homogenous the researcher should use the Fixed Effects Model.
Random effects models contemplate variability between study and within-study. The hypothesis being that studies are randomly collected and representative of various possible studies in the available literature. The researcher by using a random effects model can derive an answer to the question: “Would the treatment produce any benefit ‘on average’?” Random Effects models are considered to be “conservative” and likely to show a wider CI (confidence interval) but less likely to produce a meaningful treatment effect than a fixed effects model.
Sensitivity analysis is a replication of the Meta analysis or the primary analysis, replacing substitute conclusions or ranges of values for results that were capricious or indistinct. In simple words, it is how a researcher will judge only certain studies, groups of patients, or interventions. A sensitivity analysis provides you with an answer to the question, “Were the findings collaborative to the choices made in the course of getting them?”
When to Seek Professional Help?
You should look for people who have successfully published their meta-analysis that is similar to yours. When you are stuck in some problem, these people can be of great professional help. While some of them may charge you for their help, a lot of people are willing to help you without any charges. Those who charge also do not charge much and their assistance is worth the money. So if you see yourself being stuck in a condition that can potentially put your meta-analysis down the drain, do not hesitate to seek help from your senior colleagues.
How the Editors or Peer Reviewers evaluate your Meta Analysis
The editors, peer reviewers and later your audience will judge your Meta analysis on the following seven criteria
- Did the authors use a dedicated clinical question/hypothesis?
- Did the authors use appropriate inclusion and exclusion criteria to select articles?
- Did the authors miss one or more significant or pertinent study?
- Did the authors validate that the included studies were evaluated to ensure proper quality?
- Did the authors ensure data abstraction i.e. whether the appraisals of the studies were reproducible?
- Did the authors check for homogeneity i.e. were the results similar from study to study?
The Bottom Line:
Often we see researchers not being able to draw useful conclusions from their research projects. One of the top reasons for this failure is not being able to conduct a sound meta-analysis. There are some factors that directly influence this. We have shortlisted some of the most important things you need to keep in mind while getting your meta-analysis published. By being mindful of these important points, you are sure to avoid the common pitfalls that many researchers face while conducting their meta-analysis.
- Meta-Stat – It is free for non-commercial, educational use. It’s an on-line manual that offers step-by-step directives on the design, coding, and analysis for your meta-analysis.
- Comprehensive Meta Analysis
- Meta Analysis Made Easy
- Hasselblad V, McCrory DC. Meta-analytic tools for medical decision-making: a practical guide. Med Decis Mak 1997; 15: 81-96.