Can large language models help augment English psycholinguistic datasets?
Behav Res Methods. 2024 Jan 23. doi: 10.3758/s13428-024-02337-z. Online ahead of print.ABSTRACTResearch on language and cognition relies extensively on psycholinguistic datasets or "norms". These datasets contain judgments of lexical properties like concreteness and age of acquisition, and can be used to norm experimental stimuli, discover empirical relationships in the lexicon, and stress-test computational models. However, collecting human judgments at scale is both time-consuming and expensive. This issue of scale is compounded for multi-dimensional norms and those incorporating context. The current work asks whether la...
Source: Behavior Research Methods - January 23, 2024 Category: Psychiatry & Psychology Authors: Sean Trott Source Type: research
The effectiveness of warning statements in reducing careless responding in crowdsourced online surveys
Behav Res Methods. 2024 Jan 18. doi: 10.3758/s13428-023-02321-z. Online ahead of print.ABSTRACTCarelessness or insufficient effort responding is a widespread problem in online research, with estimates ranging from 3% to almost 50% of participants in online surveys being inattentive. While detecting carelessness has been subject to multiple studies, the factors that reduce or prevent carelessness are not as well understood. Initial evidence suggests that warning statements prior to study participation may reduce carelessness, but there is a lack of conclusive high-powered studies. This preregistered randomized controlled ex...
Source: Behavior Research Methods - January 18, 2024 Category: Psychiatry & Psychology Authors: Florian Br ühlmann Zgjim Memeti Lena F Aeschbach Sebastian A C Perrig Klaus Opwis Source Type: research
What do we manipulate when reminding people of (not) having control? In search of construct validity
Behav Res Methods. 2024 Jan 17. doi: 10.3758/s13428-023-02326-8. Online ahead of print.ABSTRACTThe construct of personal control is crucial for understanding a variety of human behaviors. Perceived lack of control affects performance and psychological well-being in diverse contexts - educational, organizational, clinical, and social. Thus, it is important to know to what extent we can rely on the established experimental manipulations of (lack of) control. In this article, we examine the construct validity of recall-based manipulations of control (or lack thereof). Using existing datasets (Study 1a and 1b: N = 627 and N = ...
Source: Behavior Research Methods - January 17, 2024 Category: Psychiatry & Psychology Authors: Marcin Bukowski Anna Potoczek Krystian Barzykowski Johannes Lautenbacher Michael Inzlicht Source Type: research
Automated scoring of the autobiographical interview with natural language processing
Behav Res Methods. 2024 Jan 17. doi: 10.3758/s13428-023-02145-x. Online ahead of print.ABSTRACTThe autobiographical interview has been used in more than 200 studies to assess the content of autobiographical memories. In a typical experiment, participants recall memories, which are then scored manually for internal details (episodic details from the central event) and external details (largely non-episodic details). Scoring these narratives requires a significant amount of time. As a result, large studies with this procedure are often impractical, and even conducting small studies is time-consuming. To reduce scoring burden...
Source: Behavior Research Methods - January 17, 2024 Category: Psychiatry & Psychology Authors: Ruben D I van Genugten Daniel L Schacter Source Type: research
Development of an open-source solution to facilitate the use of one-button wearables in experience sampling designs
Behav Res Methods. 2024 Jan 17. doi: 10.3758/s13428-023-02322-y. Online ahead of print.ABSTRACTThe experience sampling method (ESM) allows for a high degree of ecological validity compared to laboratory research, at the cost of greater effort for participants. It would therefore benefit from implementations that reduce participant effort. In the present paper, we introduce a screenless wrist-worn one-button wearable as an unobtrusive measurement method that can be employed in ESM designs. We developed an open-source Android application to make this commercially available wearable easily configurable and usable. Over the co...
Source: Behavior Research Methods - January 17, 2024 Category: Psychiatry & Psychology Authors: Selina Volsa David Lewetz Vinka Mlakic Chiara Bertagnoli Samantha Hochst öger Martina Rechl Hannah Sertic Bernad Batinic Stefan Stieger Source Type: research
Implementation of an online spacing flanker task and evaluation of its test-retest reliability using measures of inhibitory control and the distribution of spatial attention
Behav Res Methods. 2024 Jan 16. doi: 10.3758/s13428-023-02327-7. Online ahead of print.ABSTRACTThe flanker task (Eriksen & Eriksen, Perception & Psychophysics, 16(1), 143-149, 1974) has been highly influential and widely used in studies of visual attention. Its simplicity has made it popular to include it in experimental software packages and online platforms. The spacing flanker task (SFT), in which the distance between the target and flankers varies, is useful for studying the distribution of attention across space as well as inhibitory control. Use of the SFT requires that the viewing environment (e.g., stimulus...
Source: Behavior Research Methods - January 16, 2024 Category: Psychiatry & Psychology Authors: Sang Ho Lee Mark A Pitt Source Type: research
Large eye-head gaze shifts measured with a wearable eye tracker and an industrial camera
Behav Res Methods. 2024 Jan 10. doi: 10.3758/s13428-023-02316-w. Online ahead of print.ABSTRACTWe built a novel setup to record large gaze shifts (up to 140[Formula: see text]). The setup consists of a wearable eye tracker and a high-speed camera with fiducial marker technology to track the head. We tested our setup by replicating findings from the classic eye-head gaze shift literature. We conclude that our new inexpensive setup is good enough to investigate the dynamics of large eye-head gaze shifts. This novel setup could be used for future research on large eye-head gaze shifts, but also for research on gaze during e.g...
Source: Behavior Research Methods - January 10, 2024 Category: Psychiatry & Psychology Authors: Ignace T C Hooge Diederick C Niehorster Marcus Nystr öm Roy S Hessels Source Type: research
A hybrid approach to dynamic cognitive psychometrics : Dynamic cognitive psychometrics
Behav Res Methods. 2024 Jan 10. doi: 10.3758/s13428-023-02295-y. Online ahead of print.ABSTRACTDynamic cognitive psychometrics measures mental capacities based on the way behavior unfolds over time. It does so using models of psychological processes whose validity is grounded in research from experimental psychology and the neurosciences. However, these models can sometimes have undesirable measurement properties. We propose a "hybrid" modeling approach that achieves good measurement by blending process-based and descriptive components. We demonstrate the utility of this approach in the stop-signal paradigm, in which parti...
Source: Behavior Research Methods - January 10, 2024 Category: Psychiatry & Psychology Authors: Charlotte C Tanis Andrew Heathcote Mark Zrubka Dora Matzke Source Type: research
Diminished diversity-of-thought in a standard large language model
Behav Res Methods. 2024 Jan 9. doi: 10.3758/s13428-023-02307-x. Online ahead of print.ABSTRACTWe test whether large language models (LLMs) can be used to simulate human participants in social-science studies. To do this, we ran replications of 14 studies from the Many Labs 2 replication project with OpenAI's text-davinci-003 model, colloquially known as GPT-3.5. Based on our pre-registered analyses, we find that among the eight studies we could analyse, our GPT sample replicated 37.5% of the original results and 37.5% of the Many Labs 2 results. However, we were unable to analyse the remaining six studies due to an unexpec...
Source: Behavior Research Methods - January 9, 2024 Category: Psychiatry & Psychology Authors: Peter S Park Philipp Schoenegger Chongyang Zhu Source Type: research
On the reliability of value-modulated attentional capture: An online replication and multiverse analysis
Behav Res Methods. 2024 Jan 9. doi: 10.3758/s13428-023-02329-5. Online ahead of print.ABSTRACTStimuli predicting rewards are more likely to capture attention, even when they are not relevant to our current goals. Individual differences in value-modulated attentional capture (VMAC) have been associated with various psychopathological conditions in the scientific literature. However, the claim that this attentional bias can predict individual differences requires further exploration of the psychometric properties of the most common experimental paradigms. The current study replicated the VMAC effect in a large online sample ...
Source: Behavior Research Methods - January 9, 2024 Category: Psychiatry & Psychology Authors: Francisco Garre-Frutos Miguel A Vadillo Felisa Gonz ález Juan Lupi áñez Source Type: research
Short-time AOIs-based representative scanpath identification and scanpath aggregation
Behav Res Methods. 2024 Jan 9. doi: 10.3758/s13428-023-02332-w. Online ahead of print.ABSTRACTA new algorithm to identify a representative scanpath in a sample is presented and evaluated with eye-tracking data. According to Gestalt theory, each fixation of the scanpath should be on an area of interest (AOI) of the stimuli. As with existing methods, we first identify the AOIs and then extract the fixations of the representative scanpath from the AOIs. In contrast to existing methods, we propose a new concept of short-time AOI and extract the fixations of representative scanpath from the short-time AOIs. Our method outperfor...
Source: Behavior Research Methods - January 9, 2024 Category: Psychiatry & Psychology Authors: He Huang Philipp Doebler Barbara Mertins Source Type: research
Some students are more equal: Performance in Author Recognition Test and Title Recognition Test modulated by print exposure and academic background
Behav Res Methods. 2024 Jan 9. doi: 10.3758/s13428-023-02330-y. Online ahead of print.ABSTRACTReading is a key skill for university students. The Author Recognition Test (ART) and the Title Recognition Test (TRT) have both been used for decades to measure print exposure which correlates with reading and other linguistic skills. Given the available evidence for interindividual differences in reading skills, this study addresses three open issues. First, to what extent do ART and TRT scores correlate with individual differences regarding students' study programs? Second, how do these results correlate with the self-reported ...
Source: Behavior Research Methods - January 9, 2024 Category: Psychiatry & Psychology Authors: Marion Hug Julian Jarosch Christiane Eichenauer Selina Pennella Franziska Kretzschmar Pascal Nicklas Source Type: research
Diminished diversity-of-thought in a standard large language model
Behav Res Methods. 2024 Jan 9. doi: 10.3758/s13428-023-02307-x. Online ahead of print.ABSTRACTWe test whether large language models (LLMs) can be used to simulate human participants in social-science studies. To do this, we ran replications of 14 studies from the Many Labs 2 replication project with OpenAI's text-davinci-003 model, colloquially known as GPT-3.5. Based on our pre-registered analyses, we find that among the eight studies we could analyse, our GPT sample replicated 37.5% of the original results and 37.5% of the Many Labs 2 results. However, we were unable to analyse the remaining six studies due to an unexpec...
Source: Behavior Research Methods - January 9, 2024 Category: Psychiatry & Psychology Authors: Peter S Park Philipp Schoenegger Chongyang Zhu Source Type: research
Assessing the data quality of AdHawk MindLink eye-tracking glasses
In this study, a series of tests were conducted to compare the data quality of MEMS-based eye-tracking glasses (AdHawk MindLink) with three widely used camera-based eye-tracking devices (EyeLink Portable Duo, Tobii Pro Glasses 2, and SMI Eye Tracking Glasses 2). The data quality measures assessed in these tests included accuracy, precision, data loss, and system latency. The results suggest that, overall, the data quality of the eye-tracking glasses was lower compared to that of a desktop EyeLink Portable Duo eye-tracker. Among the eye-tracking glasses, the accuracy and precision of the MindLink eye-tracking glasses were e...
Source: Behavior Research Methods - January 3, 2024 Category: Psychiatry & Psychology Authors: Zehao Huang Xiaoting Duan Gancheng Zhu Shuai Zhang Rong Wang Zhiguo Wang Source Type: research
Assessing the data quality of AdHawk MindLink eye-tracking glasses
In this study, a series of tests were conducted to compare the data quality of MEMS-based eye-tracking glasses (AdHawk MindLink) with three widely used camera-based eye-tracking devices (EyeLink Portable Duo, Tobii Pro Glasses 2, and SMI Eye Tracking Glasses 2). The data quality measures assessed in these tests included accuracy, precision, data loss, and system latency. The results suggest that, overall, the data quality of the eye-tracking glasses was lower compared to that of a desktop EyeLink Portable Duo eye-tracker. Among the eye-tracking glasses, the accuracy and precision of the MindLink eye-tracking glasses were e...
Source: Behavior Research Methods - January 3, 2024 Category: Psychiatry & Psychology Authors: Zehao Huang Xiaoting Duan Gancheng Zhu Shuai Zhang Rong Wang Zhiguo Wang Source Type: research
