Anyone reading the podiatric and orthopedic literature over the years will have noticed the progressive increase in the number of published systematic reviews and meta-analyses. It makes sense that as the number of research studies has grown, so would the numbers of these types of studies.

As a quick aside, let’s recall exactly what these study types are and why they’re important, since there is a fundamental difference between them. A systematic review prospectively creates a list of criteria to include certain journal articles and exclude others with the intent to answer a research question. An organized and predetermined database search is then performed, and the results of the included studies are reviewed. This is a useful method to gather the best available evidence on a particular topic in question, with the intent to limit bias and prevent the authors from cherry picking studies to influence an answer.

---

A Systematic Review gathers the results of a pre-determined literature search, reviews and compares the results.

---

Meta-analysis is different from a systematic review in a fundamental way. After the journal articles are found, a statistical analysis is performed. This usually includes a determination of heterogeneity of the included studies (using the inconsistency index I²) and analysis of potential bias. I² values greater than 75% are considered to indicate high levels of heterogeneity¹. An analysis that pools the data from the included studies creates essentially one giant study. Clearly, the more homogeneous the studies are, the more legitimately their data can be pooled together to be analyzed. The ability to perform this type of analysis is one of the reasons meta-analysis is considered the highest level of evidence.

---

A Meta-Analysis is an analysis that pools the data from the included studies, creating essentially one giant study. It is considered the highest level of evidence.

---

That brings up an important caveat. A meta-analysis is only as strong as the studies used. For example, if the analysis includes double-blind, randomized controlled trials of high quality, then the meta-analysis would likewise contain the highest evidence and be considered Level 1 evidence. On the other hand, if the studies included are a bunch of lower level case series, then the meta-analysis could only be Level 2 evidence at best.

---

One important caveat - A meta-analysis is only as strong as the studies used.

---

Additionally, we must be aware of the ways in which these potentially useful studies can be affected by or cause interpretation errors. Publication bias is a well-known problem in the medical literature. This occurs because studies with significant, positive results are more likely to be published. The meta-analysis should include reporting on their methods to reduce publication bias. Since meta-analyses are complex, readers not educated in the complex statistical methods may misinterpret the results or attribute significance to something that should not have been included. For example, a meta-analysis that uses studies with small sample sizes is more likely to be inaccurate because those individual studies are more likely to have variance affected by chance². It is helpful, then, to use guidelines such as the PRISMA statement³, which guides readers through the important components of these reviews.

---

Publication bias - studies with significant, positive results are more likely to be published.

---

Kudos go out to the editors and staff of the Journal of Foot and Ankle Surgery, who have published several systematic reviews and meta-analyses. Take this month’s volume for example (November/December 2018). It contains three systematic reviews, one meta-analysis, and a clinical practice guideline (just threw that last one in for info’s sake).

We have been informed that sesamoidectomy for hallux sesamoid disorders yields good clinical outcomes and a high rate of return to sport, but with a 22.5% complication rate⁴ and early postoperative weightbearing for first metatarsophalangeal joint arthrodesis⁵ and Lapidus arthrodesis⁶ may be a reasonable approach with 6.35% and 3.61% nonunion rates, respectively. These three systematic reviews are correctly listed as Level 4 evidence, since most of the included studies are mostly retrospective cohort in nature. This level reflects the current generally weak state of the surgical research for these disorders and that more randomized studies are needed. Now, despite this deficiency, these studies remain useful and publishable, because they still represent the best evidence we have on these topics. As a surgeon, I find these useful as I increasingly allow my patients to bear weight earlier with few overall complications.

The meta-analysis, performed by Zhou, et al⁷ compared surgical versus nonsurgical treatment of acute Achilles ruptures, a well-known controversial subject. Looking at the criteria of the PRISMA statement, this is a high quality study that deserves its evidence Level one. The authors calculated for bias, which was low, as well as heterogeneity, which was also low. They included 10 randomized studies with a total of 934 patients. Overall, surgical repair had a lower re-rupture rate, but if an early exercise program was used, the surgical and nonsurgical re-rupture rates were equivalent. Functional outcomes were better in the surgical group, and return to sport was equivalent between both methods. Other complications, such as infection, were higher in the surgical group (which is obvious). I don’t feel this meta-analysis puts the question to bed quite yet, but it provides our best evidence yet that a functional rehabilitation program that includes early range of motion may be as good as primary surgical repair for acute Achilles ruptures.

Systematic reviews and meta-analyses clearly have their advantages and limitations, and, like all evidence-based medicine, should incorporate provider expertise and our patients’ needs and desires. Are these tests important? Yes. Are they overrated? I don’t think so…as long as we understand the limitations and potential ways these studies can create errors. With this in mind, we have a powerful way to augment the current medical literature and help us better understand how we treat patients.

Best wishes.