Question about Groups and supplying separate stimuli to each participant

PennController for IBEX Forums Support Question about Groups and supplying separate stimuli to each participant

This topic contains 7 replies, has 2 voices, and was last updated by Jeremy Jeremy 1 day, 20 hours ago.

Viewing 8 posts - 1 through 8 (of 8 total)
  • Author
    Posts
  • #5248
    Avatar
    Ncomeau
    Participant

    Hello,

    I am a research specialist setting up a speech perception/judgement experiment for my lab through pcIBEX. The experiment would demand supplying different sets of audio files to each different that takes part in the study, with no two subjects receiving the same set of files. Is using the group functionality of the CSV file (just have as many groups as the number of expected participants) we provide sufficient for doing this? Does iterating through groups work randomly or in the order supplied?

    Thank you for the help!

    #5251
    Jeremy
    Jeremy
    Keymaster

    Hello,

    In theory yes, you could define as many different groups in the CSV as you’ll have participants. Two problems though are ensuring 1) that each participant is really assigned a different group than any other participant, and 2) that you really have the number of participants you expect.

    Practically speaking, you can try to address the first point by incrementing the counter as soon as the experiment page opens. It would still take a few milliseconds though, so you might end up with some participants being assigned the same group if they take your experiment at the same time. This problem is really hard to avoid.

    The second point is particularly problematic when using recruiting platforms like MTurk or Prolific. The problem here is that several people will start taking your experiment and quit before the end. If you’re incrementing the counter whenever someone opens your experiment, those participants will mess with group assignment.

    The ideal solution would be to recruit all your participants first, assign an individual group to each of them, and then run the experiment. I don’t know if that’s an option for you. Or if you can afford it, you can run one participant at a time.

    Remember that you can force a group in the URL by using this format: https://expt.pcibex.net/ibexexps/example/example/server.py?withsquare=0 (the 0 here means “run the first group”—use 1 for the second, 2 for the third, etc.)

    Jeremy

    #5279
    Avatar
    Ncomeau
    Participant

    Hi Jeremy,

    Thank you so much for the help. I just had a quick follow up question: In an effort to retain as much information generated by a given run of the experiment as possible (and given it’s possible that people will quit early), is there a way to write out to the results file after every trial? I tried placing a SendResults() command at the end of my trials and this seems to do the trick, but there is a prompt in the browser “WARNING: Results have already been sent once. Did you forget to set the ‘manualSendResults’ config option?” that I am not sure how to suppress.

    Thanks again,

    Nickolas

    #5281
    Jeremy
    Jeremy
    Keymaster

    Hi Nickolas,

    You currently can only send results once per participant, so chances are you are seeing this message because your experiment attempts to send the results more than once.

    I might implement an option in the future to regularly send results to the server over the course of the experiment, as you suggest, but that would require revising how the server handles result-saving requests, which I can do on the PCIbex Farm but not on the original Ibex Farm, of course, and for now I am trying to keep PennController as compatible with the original farm as possible.

    Jeremy

    #5402
    Avatar
    Ncomeau
    Participant

    Hi again Jeremy,

    I was playing around with this some more last week, and I noticed that leaving a SendResults() function call at the end of each trial still writes out to the results file just fine. It doesn’t overwrite the contents that already exist for the particular run of the experiment, so there is some redundant info left but that would be easy to skip over when it comes time for analysis. Is it possible at all to, from within the main.js script, prevent the manualSendResults prompt from coming up in the browser? I know extensions and userscripts exist for disabling all prompts of this kind, but I’d really only need to stop this one from coming up and interrupting the flow of the experiment for the participant.

    Thanks again for the help with all of this!

    Nickolas

    #5404
    Jeremy
    Jeremy
    Keymaster

    Hi Nickolas,

    I hadn’t noticed that the results were sent anyway. You should have the same MD5-hased IP each time the results are sent, but a different ReceptionTime, so you won’t be able to use those to discriminate redundant results lines from the same participant form results lines from two participants taking your experiment on the same machine with the same browser. If you log a uniquely identifying ID otherwise then that’s not a problem.

    Also keep in mind that redundancy might make your results files grow quite fast, which could be a problem depending on the situation. The size of your results files counts toward calculating your quota on the PCIbex Farm, for example.

    All that being said, you can add this bit of code to your script to disable the specific prompt:

    const oldAlert = window.alert;
    window.alert = function(message) {
        if (message=="WARNING: Results have already been sent once. Did you forget to set the 'manualSendResults' config option?")
            return;
        oldAlert.call(this,message);
    }

    Jeremy

    #5463
    Avatar
    Ncomeau
    Participant

    Thank you so much, that code definitely does the job. I was doing some more thinking about how I might achieve having all of results from the participants who completed the experiment in the same results file, and was wondering if it is possible to specify a results file name from within main.js for any given call of SendResults()? I noticed in the github code for penncontroller that you can change the RESULT_FILE_NAME variable in server_conf.py (if you’re running an instance of PCIbex farm on your own server). Is there any way I can get at the equivalent variable that is pointing to the results file in my experiment overview “results” section (contains raw_results and results right now) to specify that, if it gets to the final SendResults() in the sequence, it should write them out to a different file there?

    #5464
    Jeremy
    Jeremy
    Keymaster

    Hi Nickolas,

    You won’t be able to specify a different filename for the results file as all the results are sent to the same address, and storing them in the results file is handled on the server side entirely.

    However it should be simple enough to look at the lines from your results files that correspond to your last trial so as to list all the participants who made it to the end of your experiment, and then filter the results using that list.

    Jeremy

Viewing 8 posts - 1 through 8 (of 8 total)

You must be logged in to reply to this topic.