The influence of the team in conducting a systematic review (2017) Lesley Uttley & Paul Montgomery

Open Access: https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-017-0548-x

Abstract
There is an increasing body of research documenting flaws in many published systematic reviews’ methodological and reporting conduct. When good systematic review practice is questioned, attention is rarely turned to the composition of the team that conducted the systematic review. This commentary highlights a number of relevant articles indicating how the composition of the review team could jeopardise the integrity of the systematic review study and its conclusions. Key biases require closer attention such as sponsorship bias and researcher allegiance, but there may also be less obvious affiliations in teams conducting secondary evidence-syntheses. The importance of transparency and disclosure are now firmly on the agenda for clinical trials and primary research, but the meta-biases that systematic reviews may be at risk from now require further scrutiny.

Seems relevant, considering we've got a Cochrane review co-authored by White coming up.

 

So much of this was relevant to us that it felt silly to pick bits out, but to try to give a flavour of it I've tried to select the 20% most likely to be of interest.

"They can be susceptible to bias, for example, when reviews are conducted by people who have a stake in the conclusions (researcher allegiance) [10, 11]. Alternatively, reviews could be conducted carelessly, in the chosen methods of meta-analysis or study selection (meta-bias) [12], or by failing to report research misconduct even when identified [13]. If flawed systematic reviews continue to be published, they risk losing their eminent position in the evidence hierarchy; therefore, closer examination of who might be conducting them, and how the output can be affected, is warranted."

"But a more troubling motive is that systematic reviews may be conducted and published by people who have a clear stake in publishing positive results. How can we know if the authors of a systematic review are affiliated to the review question? Despite the existence of guidelines and reporting standards for systematic reviews, it is not currently a requirement to declare the motives of the reviewers, i.e., who is behind the origination of the review question (except for sponsored work) and whether the same people who set the review question are also involved with answering the review question. Information about whether the research question originated from outside the review team, such as through a commissioned call, are not currently deemed as relevant to potential assessment of bias in systematic reviews."

- White provided some funding for the development of the Cochrane IPD protocol. I've no idea where the idea for it originated.

"However, even when methods are stated a priori in systematic review protocols, deviations from the protocol, whether justified or not, may be seldom reported in the final publication, as highlighted by Silagy et al. [16] who found 43 out of a sample of 47 published Cochrane reviews had undergone a major change compared with the most recently published protocol."

"From protocol design to analysis, there are opportunities for team members to shape the project and these individual influences can affect the output [22]. Biases may be fairly obvious such as highlighting the most favourable results or perhaps more subtle and hard to detect such as biases in study selection (into the review or into the meta-analysis) towards positive studies. The motives of the review team can influence whether research findings represent true treatment effects and are able to be replicated [23] such as financial or other interests within the review team [12]. Ebrahim et al. [24] conducted a study of 185 meta-analyses in antidepressants to find that 29% of papers contained authors who were employees of the assessed drug manufacturer and that 79% had an industry link to the drug assessed. This study also found that meta-analyses including an author who was an employee of the manufacturer of the assessed drug were 22 times less likely to have negative statements about the drug than other meta-analyses."

"However, standard CoI statements often focus on narrow commercial interests and may be inadequate to reveal potentially hidden agendas [27]. Consumers of systematic reviews cannot rely solely on declarations of competing interests, which may relate to recent pecuniary funding (within the last 3 years), as opposed to more long-term affiliations to health interventions, to know whether those conducting the review have an interest in the results of the research (researcher allegiance) [28]. For example, for some, their very employment is reliant on a given intervention’s reputation such as homoeopaths or psychotherapists and they are unsurprisingly unlikely to publish a rigorous review with neutral or negative conclusions underpinning the basis of their profession. A recent study investigating the relationship between CoI and the conclusions of systematic reviews of psychological therapies found that “non-financial CoI” and particularly the inclusion of own primary studies into reviews were frequently seen in systematic reviews of psychological therapies [29]. Moreover, author allegiance to psychological therapy was never disclosed in 15 out of 95 reviews. Similarly, despite the apparent benefit that topic experts may bring to the review team, Gotzsche and Ioannidis (2012) [30] point out that the strong opinions of content area experts, such as clinical experts, can make it difficult to perform unbiased systematic reviews. For example, they assert that “people who have an interest in concealing uncomfortable evidence, clinicians, for example, find it particularly difficult to acknowledge the harms their interventions may cause”. Current methods for describing who is involved in the conduct of systematic reviews and their level of affiliation may occupy insufficient attention in final peer-reviewed journal publications."

"Objective research, such as secondary evidence syntheses, should not necessarily be carried out dispassionately. The contribution of user perspectives and lay-expert engagement in research are now well-recognised [31]. Without patient and public involvement, due consideration to whether the review question or the included evidence are patient-centred may not be given [32]."

"Review teams preoccupied with critiquing heterogeneity of the existing evidence may be unlikely to elaborate on the external validity of the review, e.g., by searching and critiquing relevant grey literature, or service-user perspectives. Team experience or expertise could affect the external validity of a systematic review therefore. For example, research indicates that few strategies in the Cochrane Collaboration explicitly address the research priorities of disadvantaged populations and innovative approaches are needed to ensure that the research priorities of diverse stakeholders are considered [38, 39]. Equitable representation of population demographics within the team, as well as the more obviously required skills and experience may potentially influence the importance and uptake of the review."