Which Academic Search Systems are Suitable for Systematic Reviews or Meta ‐Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed and 26 other Resources

This study investigates and compares the systematic search qualities of 28 widely used academic search systems, including Google Scholar, PubMed and Web of Science. A novel, query ‐based method tests how well users are able to interact and retrieve records with each system. The study is the first to show the extent to which search systems can effectively and efficiently perform (Boolean) searches with regards to precision, recall and reproducibility. We found substantial di fferences in the performance of search systems, meaning that their usability in systematic searches varies. Indeed, only half of the search systems analysed and only a few Open Access databases can be recommended for evidence syntheses without adding substantial caveats. Particularly, our findings d emonstrate why Google Scholar is inappropriate as principal search system.We call for database owners to recognise the requirements of evidence synthesis, and for academic journals to re ‐assess quality requirements for systematic reviews. Our findings aim to support researchers in conducting better searches for better evidence synthesis.
Source: Research Synthesis Methods - Category: Chemistry Authors: Tags: RESEARCH ARTICLE Source Type: research