Accuracy of reaction times recorded by pcIBEX

[mschrumpf]

Hello everyone.
Is there any data available on the accuracy of reaction times measured in pcIBEX online studies?
Our research group is currently preparing a self-paced reading (dashed sentences) experiment. We expect to have a difference of around 25 milliseconds between testing conditions. I am wondering whether any research has been conducted on how precise pcIBEX is in recording reaction times when compared to other software/programming environments (e.g. PsychoPy). This would be essential for us to know since we expect to be dealing with very small differences.
Thank you in advance for your help.

[Jeremy]

Hi,

I’m not aware of any paper systematically assessing the accuracy of reaction times in Ibex (or PCIbex) but I wouldn’t be surprised that some published papers reported self-paced-reading experiments designed with Ibex.

What I can tell you though is that you shouldn’t worry about varying speeds of the Internet connections: at least with (PC)Ibex, but I suspect other online softwares do the same thing, all the content of the experiment is downloaded at the beginning (modulo images/audios/etc. in pure Ibex–PCIbex does handle media preloading). So once a participant starts an experiment, they won’t need to communicate with the server until they submit their results at the very end.

The concerns mostly come from each participant’s local environment: do they have lots of tabs open in their browser? Do they have an old, weak processor? What kind of keyboard are they using? If all goes well, most participants won’t encounter performance issues, and then maybe you’ll be able to detect an effect size of 25ms.

If you can afford it, maybe run a small control experiment where you know there to be an effect of comparable size between two conditions to get a sense of what you’ll get?

Hopefully other people here can point you to relevant papers

Jeremy

[florians]

I’m not aware off the top of my head of work specifically looking at the accuracy of (PC)Ibex response/reaction times, but we’ve had a good bit of work in our department on priming in David Embick’s lab that has been able to find some pretty small effect sizes (definitely < 100ms), e.g. - https://www.ling.upenn.edu/~embick/wilder-etal-18.pdf
- http://www.avacreemers.com/wp-content/uploads/2020/04/Creemers_dissertation_April10.pdf (a dissertation that was just defended today; see, e.g., chapter 5 studies on priming in English compounds that were all run on IBEX using PennController; pre- PCIbex farm, but should be fully comparable).

Not self-paced reading, but from the technical side in terms of RT accuracy, that shouldn't be relevant.

Florian

[mschrumpf]

Thank you two very much. Your answers were both very helpful. I am going to discuss the matter with my team.

[Author]

Posts