UW Shoulder and Elbow Academy: Spinning the data on the value of glenoid augmentation

In the previous post we discussed the challenges in doing clinically significant research on shoulder arthroplasty.

As a follow-on, here we focus on the importance of the way research is presented, with particular reference to spin – defined as bias that overstates efficacy and/or underestimates harms of a treatment. Systematic reviews and meta-analyses are at risk for spin if there was bias in the primary studies on which they were based. 

There is the potential for spin in any presentation of outcomes. Evaluation of spin in reviews of biodegradable balloon spacers for massive irreparable rotator cuff tears found that 93.1% of the 29 included studies had at least one type of spin. See other examples of spin in our literature on this post. 

A recent article provides a useful guide to the elements of spin and how to avoid them.

The authors of Evaluation of Spin in Systematic Reviews and Meta-Analyses Involving Glenoid Augmentation in Total Shoulder Arthroplasty assessed the quantity and types of spin in systematic reviews and meta-analyses of glenoid augmentation in shoulder arthroplasty. They searched for each of 15 types of spin (see A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity). At least one form of spin was identified in 13 (81.3%) of the 16 studies. 

“The conclusion claims the beneficial effect of the experimental  treatment despite a high risk of bias in primary studies” was the most commonly occurring type of spin in this review; it is found in many previous studies in other orthopaedic literature, ranging from 23.1%-65%. A common weakness contributing to this type spin was drawing conclusions based on primary studies of low levels of evidence

‘‘Conclusion claims the beneficial effect of the experimental treatment despite reporting bias’’ was the next most common; which may mislead readers by the selective inclusion and omission of results in the abstract. Reporting bias results from the tendency to overreport or selectively publish positive results. One example of reporting bias can be seen in a recent article that concluded that the reverse shoulder arthroplasty “provided highly favorable results” but only reported the statistically significant improvement in Constant scores and omitted the lack of statistically significant improvements in VAS, ASES,  SST, and functional range of motion measurements. 

The 15 types of spin are listed here as a heads up for surgeons considering publishing a systematic review. 

The title claims or suggests a beneficial effect of the experimental intervention not supported by the findings 

Authors hide or do not present any conflict of interest 

Selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention 

Selective reporting of or overemphasis on harm outcomes or analysis favoring the safety of the experimental intervention 

Failure to specify the direction of the effect when it favors the control intervention 

Failure to report a wide confidence interval of estimates 

The conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies – Most common 43.8 % of the glenoid augmentation studies.

The conclusion claims the beneficial effect of the experimental treatment despite reporting bias. Second most common (37.5% of the glenoid augmentation studies.).

The conclusion formulates recommendations for clinical practice not supported by the findings 

The conclusion claims safety based on non-statistically significant results with a wide confidence interval 

The conclusion focuses selectively on statistically significant efficacy outcome 

The conclusion claims equivalence or comparable effectiveness for non-statistically significant results with a wide confidence interval 

The conclusion extrapolates the review findings to a different intervention (e.g., claiming efficacy of one specific intervention although the review covered a class of several interventions) 

Conclusion extrapolates the review’s findings from a surrogate marker or a specific outcome to the global improvement of the disease 

Conclusion extrapolates the review’s findings to a different population or setting 

The authors also applied A Measurement Tool to Assess Systematic Reviews (AMSTAR 2), a questionnaire that quantifies the quality of a systematic  review based on criteria such as whether authors reported presence of bias, impact of bias, the use of a predetermined protocol, funding sources, and conflicts of interest, and/or adequately characterized studies included in the review. Based on this review three (18.8%) of the studies were related as “moderate” quality and the remaining thirteen (81.3%) were rated as “low” quality. None met the criteria for “high” quality.  The elements of the AMSTAR 2 are shown below

Did the research questions and inclusion criteria for the review include the elements of PICO (Patient, Population, or Problem; Intervention; Comparison; Outcome)? 

Did the report of the review contain an explicit statement that the review methods were established before the conduct of the review, and did the report justify any significant deviations from the protocol?
Did the review authors explain their selection of the study designs for inclusion in the review?
Did the review authors use a comprehensive literature search strategy?
Did the review authors perform study selection in duplicate?
Did the review authors perform data extraction in duplicate?
Did the review authors provide a list of excluded studies and justify the exclusions?
Did the review authors describe the included studies in adequate detail?
Did the review authors use a satisfactory technique for assessing the risk of bias in individual studies that were included in the review? 

Did the review authors report on the sources of funding for the studies included in the review?

The authors also found a statistically significant association between the presence of a conflict of interest and the lack of reporting funding sources. These conflicts included examples of all of the following: authors who reported receiving grants, personal fees, royalties, and research fees from orthopedic device manufacturers, as well as authors who were investors, presenters, or consultants for orthopedic device manufacturers.

They concluded  that “Spin is highly prevalent in the abstracts of systematic reviews and meta-analyses studying glenoid augmentation with TSA. Misleading reporting is the most common category of spin.“

We want our publications to be as useful and as transparent as possible. Hopefully, this guide will help us avoid spin when we present our work.

Bullock’s Oriole

Umtanum Washington, 5/25/25