Experiment commensurability does not necessitate research consolidation
Behav Brain Sci. 2024 Feb 5;47:e61. doi: 10.1017/S0140525X23002364.ABSTRACTIntegrative experiment design promises to foster cumulative knowledge by changing how we design experiments, build theories, and conduct research. I support the push to increase commensurability across experimental research but raise several reservations regarding results-driven and large-team-based research. I argue that it is vital to preserve academic diversity and adversarial debate via independent efforts.PMID:38311441 | DOI:10.1017/S0140525X23002364 (Source: The Behavioral and Brain Sciences)
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: Milena Tsvetkova Source Type: research
The miss of the framework
Behav Brain Sci. 2024 Feb 5;47:e59. doi: 10.1017/S0140525X23002315.ABSTRACTThe authors rightly critique existing social sciences approaches. However, they are too quick to dismiss the criticism that their proposed paradigm is atheoretical. Social and cognitive theories are indeed incommensurate, often due to the lack of a unifying framework. Without proper integration with theoretical frameworks, their proposal may merely produce a resource-intensive veneer of thoroughness without substantive improvements to understanding.PMID:38311442 | DOI:10.1017/S0140525X23002315 (Source: The Behavioral and Brain Sciences)
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: Paul E Smaldino Source Type: research
The future of experimental design: Integrative, but is the sample diverse enough?
Behav Brain Sci. 2024 Feb 5;47:e42. doi: 10.1017/S0140525X23002212.ABSTRACTAlmaatouq et al. propose an "integrative approach" to increase the generalisability and commensurability of experiments. Yet their metascientific approach has one glaring omission (and misinterpretation of) - the role of sample diversity in generalisability. In this commentary, we challenge false notions of subsumed duality between contexts, population, and diversity, and propose modifications to their design space to accommodate sample diversity.PMID:38311443 | DOI:10.1017/S0140525X23002212 (Source: The Behavioral and Brain Sciences)
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: Sakshi Ghai Sanchayan Banerjee Source Type: research
Discovering the unknown unknowns of research cartography with high-throughput natural description
Behav Brain Sci. 2024 Feb 5;47:e50. doi: 10.1017/S0140525X23002170.ABSTRACTTo succeed, we posit that research cartography will require high-throughput natural description to identify unknown unknowns in a particular design space. High-throughput natural description, the systematic collection and annotation of representative corpora of real-world stimuli, faces logistical challenges, but these can be overcome by solutions that are deployed in the later stages of integrative experiment design.PMID:38311444 | DOI:10.1017/S0140525X23002170 (Source: The Behavioral and Brain Sciences)
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: Tanay Katiyar Jean-Fran çois Bonnefon Samuel A Mehr Manvir Singh Source Type: research
Against na ïve induction from experimental data
Behav Brain Sci. 2024 Feb 5;47:e51. doi: 10.1017/S0140525X2300211X.ABSTRACTThis commentary argues against the indictment of current experimental practices such as piecemeal testing, and the proposed integrated experiment design (IED) approach, which we see as yet another attempt at automating scientific thinking. We identify a number of undesirable features of IED that lead us to believe that its broad application will hinder scientific progress.PMID:38311445 | DOI:10.1017/S0140525X2300211X (Source: The Behavioral and Brain Sciences)
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: David Kellen Gregory E Cox Chris Donkin John C Dunn Richard M Shiffrin Source Type: research
Consensus meetings will outperform integrative experiments
Behav Brain Sci. 2024 Feb 5;47:e56. doi: 10.1017/S0140525X23002248.ABSTRACTWe expect that consensus meetings, where researchers come together to discuss their theoretical viewpoints, prioritize the factors they agree are important to study, standardize their measures, and determine a smallest effect size of interest, will prove to be a more efficient solution to the lack of coordination and integration of claims in science than integrative experiments.PMID:38311446 | DOI:10.1017/S0140525X23002248 (Source: The Behavioral and Brain Sciences)
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: Maximilian A Primbs Leonie A Dudda Pia K Andresen Erin M Buchanan Hannah K Peetz Miguel Silan Dani ël Lakens Source Type: research
Integrative experiments require a shared theoretical and methodological basis
Behav Brain Sci. 2024 Feb 5;47:e34. doi: 10.1017/S0140525X2300225X.ABSTRACTCreating an integrated design space can be successful only if researchers agree on how to define and measure a certain phenomenon of interest. Adversarial collaborations and mathematical modeling can aid in reaching the necessary level of agreement when researchers depart from different theoretical perspectives.PMID:38311447 | DOI:10.1017/S0140525X2300225X (Source: The Behavioral and Brain Sciences)
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: Pietro Amerio Nicolas Coucke Axel Cleeremans Source Type: research
Explore your experimental designs and theories before you exploit them!
Behav Brain Sci. 2024 Feb 5;47:e40. doi: 10.1017/S0140525X23002303.ABSTRACTIn many areas of the social and behavioral sciences, the nature of the experiments and theories that best capture the underlying constructs are themselves areas of active inquiry. Integrative experiment design risks being prematurely exploitative, hindering exploration of experimental paradigms and of diverse theoretical accounts for target phenomena.PMID:38311449 | DOI:10.1017/S0140525X23002303 (Source: The Behavioral and Brain Sciences)
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: Marina Dubova Sabina J Sloman Ben Andrew Matthew R Nassar Sebastian Musslick Source Type: research
Representative design: A realistic alternative to (systematic) integrative design
Behav Brain Sci. 2024 Feb 5;47:e48. doi: 10.1017/S0140525X23002200.ABSTRACTWe disagree with Almaatouq et al. that no realistic alternative exists to the "one-at-a-time" paradigm. Seventy years ago, Egon Brunswik introduced representative design, which offers a clear path to commensurability and generality. Almaatouq et al.'s integrative design cannot guarantee the external validity and generalizability of results which is sorely needed, while representative design tackles the problem head on.PMID:38311450 | DOI:10.1017/S0140525X23002200 (Source: The Behavioral and Brain Sciences)
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: Gijs A Holleman Mandeep K Dhami Ignace T C Hooge Roy S Hessels Source Type: research
The elephant's other legs: What some sciences actually do
Behav Brain Sci. 2024 Feb 5;47:e35. doi: 10.1017/S0140525X23002108.ABSTRACTIntegrative experiments, as described, seem blindly empirical, as if the question of generality of effects could not be understood through controlled one-at-a-time experiments. But current research using such experiments, especially applied research, can resolve issues and make progress through understanding of cause-effect pathways, leaving to engineers the task of translating this understanding into practice.PMID:38311452 | DOI:10.1017/S0140525X23002108 (Source: The Behavioral and Brain Sciences)
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: Jonathan Baron Source Type: research
Sampling complex social and behavioral phenomena
Behav Brain Sci. 2024 Feb 5;47:e55. doi: 10.1017/S0140525X23002327.ABSTRACTWe comment on the limits of relying on prior literature when constructing the design space for an integrative experiment; the adaptive nature of social and behavioral phenomena and the implications for the use of theory and modeling when constructing the design space; and on the challenges of measuring random errors and lab-related biases in measurement without replication.PMID:38311453 | DOI:10.1017/S0140525X23002327 (Source: The Behavioral and Brain Sciences)
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: Henrik Olsson Mirta Galesic Source Type: research
Commensurability engineering is first and foremost a theoretical exercise
Behav Brain Sci. 2024 Feb 5;47:e63. doi: 10.1017/S0140525X23002224.ABSTRACTI provide a personal perspective on metastudies and emphasize lesser-known benefits. I stress the need for integrative theories to establish commensurability between experiments. I argue that mathematical social scientists should be engaged to develop integrative theories, and that likelihood functions provide a common mathematical framework across experiments. The development of quantitative theories promotes commensurability engineering on a larger scale.PMID:38311454 | DOI:10.1017/S0140525X23002224 (Source: The Behavioral and Brain Sciences)
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: Joachim Vandekerckhove Source Type: research
Integrative design for thought-experiments
Behav Brain Sci. 2024 Feb 5;47:e39. doi: 10.1017/S0140525X23002340.ABSTRACTIntegrative experiment design should be extended to thought-experiments. Thought-experiments are closely connected to "real" experiments. They are involved in devising the design space of theories and possible experiments. The latter may be partitioned into experiments to be really performed and mere thought-experiments. The proposed extension of integrative experiment design lends guidance to a more methodical performance of thought-experiments.PMID:38311455 | DOI:10.1017/S0140525X23002340 (Source: The Behavioral and Brain Sciences)
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: Daniel Dohrn Angelica Mezzadri Source Type: research
Individual differences do matter
Behav Brain Sci. 2024 Feb 5;47:e43. doi: 10.1017/S0140525X2300242X.ABSTRACTThe integrative experiment design proposal currently only relates to group results, but downplays individual differences between participants, which may nevertheless be substantial enough to constitute a relevant dimension in the design space. Excluding the individual participant in the integrative design will not solve all problems mentioned in the target article, because averaging results may obscure the underlying mechanisms.PMID:38311456 | DOI:10.1017/S0140525X2300242X (Source: The Behavioral and Brain Sciences)
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: Stefan Glasauer Source Type: research
Replies to commentaries on beyond playing 20 questions with nature
Behav Brain Sci. 2024 Feb 5;47:e65. doi: 10.1017/S0140525X23002789.ABSTRACTCommentaries on the target article offer diverse perspectives on integrative experiment design. Our responses engage three themes: (1) Disputes of our characterization of the problem, (2) skepticism toward our proposed solution, and (3) endorsement of the solution, with accompanying discussions of its implementation in existing work and its potential for other domains. Collectively, the commentaries enhance our confidence in the promise and viability of integrative experiment design, while highlighting important considerations about how it is used.P...
Source: The Behavioral and Brain Sciences - February 4, 2024 Category: Neuroscience Authors: Abdullah Almaatouq Thomas L Griffiths Jordan W Suchow Mark E Whiting James Evans Duncan J Watts Source Type: research
