Advancing unanchored simulated treatment comparisons: A novel implementation and simulation study
AbstractPopulation-adjusted indirect comparisons, developed in the 2010s, enable comparisons between two treatments in different studies by balancing patient characteristics in the case where individual patient-level data (IPD) are available for only one study. Health technology assessment (HTA) bodies increasingly rely on these methods to inform funding decisions, typically using unanchored indirect comparisons (i.e., without a common comparator), due to the need to evaluate comparative efficacy and safety for single-arm trials. Unanchored matching-adjusted indirect comparison (MAIC) and unanchored simulated treatment com...
Source: Research Synthesis Methods - April 9, 2024 Category: Chemistry Authors: Shijie Ren,
Sa Ren,
Nicky J. Welton,
Mark Strong Tags: RESEARCH ARTICLE Source Type: research
Combining randomized and non ‐randomized data to predict heterogeneous effects of competing treatments
AbstractSome patients benefit from a treatment while others may do so less or do not benefit at all. We have previously developed a two-stage network meta-regression prediction model that synthesized randomized trials and evaluates how treatment effects vary across patient characteristics. In this article, we extended this model to combine different sources of types in different formats: aggregate data (AD) and individual participant data (IPD) from randomized and non-randomized evidence. In the first stage, a prognostic model is developed to predict the baseline risk of the outcome using a large cohort study. In the secon...
Source: Research Synthesis Methods - March 20, 2024 Category: Chemistry Authors: Konstantina Chalkou,
Tasnim Hamza,
Pascal Benkert,
Jens Kuhle,
Chiara Zecca,
Gabrielle Simoneau,
Fabio Pellegrini,
Andrea Manca,
Matthias Egger,
Georgia Salanti Tags: RESEARCH ARTICLE Source Type: research
Development of a search filter to retrieve reports of interrupted time series studies from MEDLINE and PubMed
ConclusionOur filters strike different balances between comprehensiveness and screening workload and suit different research needs. Retrieval of ITS studies would be improved if authors identified the ITS design in the titles. (Source: Research Synthesis Methods)
Source: Research Synthesis Methods - March 18, 2024 Category: Chemistry Authors: Phi ‐Yen Nguyen,
Joanne E. McKenzie,
Simon L. Turner,
Matthew J. Page,
Steve McDonald Tags: RESEARCH ARTICLE Source Type: research
Can large language models replace humans in systematic reviews? Evaluating GPT ‐4's efficacy in screening and extracting data from peer‐reviewed and grey literature in multiple languages
AbstractSystematic reviews are vital for guiding practice, research and policy, although they are often slow and labour-intensive. Large language models (LLMs) could speed up and automate systematic reviews, but their performance in such tasks has yet to be comprehensively evaluated against humans, and no study has tested Generative Pre-Trained Transformer (GPT)-4, the biggest LLM so far. This pre-registered study uses a “human-out-of-the-loop” approach to evaluate GPT-4's capability in title/abstract screening, full-text review and data extraction across various literature types and languages. Although GPT-4 had accur...
Source: Research Synthesis Methods - March 15, 2024 Category: Chemistry Authors: Qusai Khraisha,
Sophie Put,
Johanna Kappenberg,
Azza Warraitch,
Kristin Hadfield Tags: RESEARCH ARTICLE Source Type: research
Easy ‐Amanida: An R Shiny application for the meta‐analysis of aggregate results in clinical metabolomics using Amanida and Webchem
This article describes all the steps to perform the meta-analysis using Easy-Amanida, including an illustrative example for interpreting the results. The use of aggregate statistics metrics extends the use of Easy-Amanida beyond the metabolomics field. (Source: Research Synthesis Methods)
Source: Research Synthesis Methods - March 14, 2024 Category: Chemistry Authors: Maria Llambrich,
Pau Satorra,
Eudald Correig,
Josep Gum à,
Jesús Brezmes,
Cristian Tebé,
Raquel Cumeras Tags: SOFTWARE FOCUS Source Type: research
LFK index does not reliably detect small ‐study effects in meta‐analysis: A simulation study
AbstractTheLFK index has been promoted as an improved method to detect bias in meta-analysis. Putatively, its performance does not depend on the number of studies in the meta-analysis. We conducted a simulation study, comparing theLFK index test to three standard tests for funnel plot asymmetry in settings with smaller or larger group sample sizes. In general, false positive rates of theLFK index test markedly depended on the number and size of studies as well as the between-study heterogeneity with values between 0% and almost 30%. Egger's test adhered well to the pre-specified significance level of 5% under homogeneity, ...
Source: Research Synthesis Methods - March 12, 2024 Category: Chemistry Authors: Guido Schwarzer,
Gerta R ücker,
Cristina Semaca Tags: RESEARCH ARTICLE Source Type: research
Issue Information
(Source: Research Synthesis Methods)
Source: Research Synthesis Methods - March 8, 2024 Category: Chemistry Tags: ISSUE INFORMATION Source Type: research
Correction to “Network Meta‐Interpolation: Effect modification adjustment in network meta‐analysis using subgroup analyses”
(Source: Research Synthesis Methods)
Source: Research Synthesis Methods - March 4, 2024 Category: Chemistry Tags: CORRECTION Source Type: research
Data extraction for evidence synthesis using a large language model: A proof ‐of‐concept study
The objective of this proof-of-concept study was to assess the performance of an LLM (Claude 2) in extracting data elements from published studies, compared with human data extraction as employed in systematic reviews. Our analysis utilized a convenience sample of 10 English-language, open-access publications of randomized controlled trials included in a single systematic review. We selected 16 distinct types of data, posing varying degrees of difficulty (160 data elements across 10 studies). We used the browser version of Claude 2 to upload the portable document format of each publication and then prompted the model for e...
Source: Research Synthesis Methods - March 3, 2024 Category: Chemistry Authors: Gerald Gartlehner,
Leila Kahwati,
Rainer Hilscher,
Ian Thomas,
Shannon Kugley,
Karen Crotty,
Meera Viswanathan,
Barbara Nussbaumer ‐Streit,
Graham Booth,
Nathaniel Erskine,
Amanda Konet,
Robert Chew Tags: RESEARCH ARTICLE Source Type: research
A comprehensive review and shiny application on the matching ‐adjusted indirect comparison
AbstractPopulation-adjusted indirect comparison (PAIC) is an increasingly used technique for estimating the comparative effectiveness of different treatments for the health technology assessments when head-to-head trials are unavailable. Three commonly used PAIC methods include matching-adjusted indirect comparison (MAIC), simulated treatment comparison (STC), and multilevel network meta-regression (ML-NMR). MAIC enables researchers to achieve balanced covariate distribution across two independent trials when individual participant data are only available in one trial. In this article, we provide a comprehensive review of ...
Source: Research Synthesis Methods - February 22, 2024 Category: Chemistry Authors: Ziren Jiang,
Joseph C. Cappelleri,
Margaret Gamalo,
Yong Chen,
Neal Thomas,
Haitao Chu Tags: SOFTWARE FOCUS Source Type: research
Estimating the extent of selective reporting: An application to economics
AbstractUsing a sample of 70,399 publishedp-values from 192 meta-analyses, we empirically estimate the counterfactual distribution ofp-values in the absence of any biases. Comparing observedp-values with counterfactually expectedp-values allows us to estimate how manyp-values are published as being statistically significant when they should have been published as non-significant. We estimate the extent of selectively reportedp-values to range between 57.7% and 71.9% of the significantp-values. The counterfactualp-value distribution also allows us to assess shifts ofp-values along the entire distribution of publishedp-value...
Source: Research Synthesis Methods - February 21, 2024 Category: Chemistry Authors: Stephan B. Bruns,
Teshome K. Deressa,
T. D. Stanley,
Chris Doucouliagos,
John P. A. Ioannidis Tags: RESEARCH ARTICLE Source Type: research
Impact of trial attrition rates on treatment effect estimates in chronic inflammatory diseases: A meta ‐epidemiological study
The objective of this meta-epidemiological study was to explore the impact of attrition rates on treatment effect estimates in randomised trials of chronic inflammatory diseases (CID) treated with biological and targeted synthetic disease-modifying drugs. We sampled trials from Cochrane reviews. Attrition rates and primary endpoint results were retrieved from trial publications; Odds ratios (ORs) were calculated from the odds of withdrawing in the experimental intervention compared to the control comparison groups (i.e., differential attrition), as well as the odds of achieving a clinical response (i.e., the trial outcome)...
Source: Research Synthesis Methods - February 14, 2024 Category: Chemistry Authors: Silja H. Overgaard,
Caroline M. Moos,
John P. A. Ioannidis,
George Luta,
Johannes I. Berg,
Sabrina M. Nielsen,
Vibeke Andersen,
Robin Christensen Tags: RESEARCH ARTICLE Source Type: research
Meta ‐analyses of partial correlations are biased: Detection and solutions
AbstractWe demonstrate that all meta-analyses of partial correlations are biased, and yet hundreds of meta-analyses of partial correlation coefficients (PCCs) are conducted each year widely across economics, business, education, psychology, and medical research. To address these biases, we offer a new weighted average, UWLS+3. UWLS+3 is the unrestricted weighted least squares weighted average that makes an adjustment to the degrees of freedom that are used to calculate partial correlations and, by doing so, renders trivial any remaining meta-analysis bias. Our simulations also reveal that these meta-analysis biases are sma...
Source: Research Synthesis Methods - February 12, 2024 Category: Chemistry Authors: T. D. Stanley,
Hristos Doucouliagos,
Tomas Havranek Tags: RESEARCH ARTICLE Source Type: research
Footprint of publication selection bias on meta ‐analyses in medicine, environmental sciences, psychology, and economics
AbstractPublication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bi...
Source: Research Synthesis Methods - February 8, 2024 Category: Chemistry Authors: Franti šek Bartoš,
Maximilian Maier,
Eric‐Jan Wagenmakers,
Franziska Nippold,
Hristos Doucouliagos,
John P. A. Ioannidis,
Willem M. Otte,
Martina Sladekova,
Teshome K. Deresssa,
Stephan B. Bruns,
Daniele Fanelli,
T. D. Stanley Tags: RESEARCH ARTICLE Source Type: research
Bayesian meta ‐analysis for evaluating treatment effectiveness in biomarker subgroups using trials of mixed patient populations
AbstractDuring drug development, evidence can emerge to suggest a treatment is more effective in a specific patient subgroup. Whilst early trials may be conducted in biomarker-mixed populations, later trials are more likely to enroll biomarker-positive patients alone, thus leading to trials of the same treatment investigated in different populations. When conducting a meta-analysis, a conservative approach would be to combine only trials conducted in the biomarker-positive subgroup. However, this discards potentially useful information on treatment effects in the biomarker-positive subgroup concealed within observed treatm...
Source: Research Synthesis Methods - February 6, 2024 Category: Chemistry Authors: Lorna Wheaton,
Dan Jackson,
Sylwia Bujkiewicz Tags: RESEARCH ARTICLE Source Type: research
