Can we make use of perception of questions' easiness in Delphi-like studies? Some results from an experiment with an alternative feedback

Publication date: Available online 5 January 2019Source: Technological Forecasting and Social ChangeAuthor(s): Carlos Tadao Kawamoto, James Terence Coulter Wright, Renata Giovinazzo Spers, Daniel Estima de CarvalhoAbstractThe main goal of this study is to investigate an alternative feedback for Delphi-like applications, based on a subgroup of elite panellists selected from their perception of questions' easiness, in order to increase the results' accuracy. The use of declared easiness perception is an old idea appearing in the earliest studies by Rand Corporation members, but rarely explored in recent publications. We ran an experiment with 79 students and 43 questions in 2015 and 2016, where the treatment subgroup received the assembled elite feedback, and the control subgroup received statistics based on all panellists. The results did not show a statistically significant predominance of higher accuracy with feedback based on the elite subgroup, although correlation between accuracy and easiness perception was not negligible.
Source: Technological Forecasting and Social Change - Category: Science Source Type: research