Catch Trials

PennController for IBEX Forums Support Catch Trials

Viewing 15 posts - 1 through 15 (of 15 total)
  • Author
    Posts
  • #5577
    Ncomeau
    Participant

    Up until this point, I have successfully been using the randomize() function inside my trial sequence to structure my experiment based on the values provided in my CSV file. My task is one of auditory perception though, so it will be important to have catch trials for determining whether or not the participant was paying attention throughout the experiment. Has anyone found a way to feasibly inject catch trials into “blocks” of the randomized set of normal trials from within the main.js script or is that something that needs to be handled when generating the CSV externally?

    #5578
    Jeremy
    Keymaster

    Hi,

    There are a couple functions you might be interested in looking up, like sepWith or randomizeNoMoreThan, but both require specific labeling.

    The way I understand your case, you have one block of undifferentiated trials in which you want to inject some other trials at regular intervals. This is definitely something you can do by coding your own function, the way I did for randomizeNoMoreThan above, once you’ve determined how frequently you want the catch trials to appear, and whether you want to re-use the same catch trial each time, or pre-generate multiple catch trials

    Another method would be to modify the running order. PennController uses this method in a few cases, but hopefully that shouldn’t create any conflicts

    Jeremy

    #5596
    Ncomeau
    Participant

    Hi Jeremy,

    I was able to get Pause messages inserted at the proper interval given those instructions on modifying the running order, but now I’m having trouble getting a trial to replace the messages. Is it possible to just perform the same ro[i].push but with a newTrial element instead of html?

    Thanks again for the help!

    Nickolas

    #5598
    Jeremy
    Keymaster

    Hi Nickolas,

    You can replace this bit of code from the documentation:

    new DynamicElement(
        "Message",
        { html: "<p>Pause</p>", transfer: 1000 },
        true
    )

    with this:

    new DynamicElement(
        "PennController",
        newTrial(
            newButton("Hello world").print().wait()
        ),
        true
    )

    You can code the newTrial as you would any regular PennController trial

    Jeremy

    #5626
    Ncomeau
    Participant

    Hi again Jeremy,

    I have been successfully using modifyRunningOrder() along with the other code you provided above to insert catch trials at a regular interval, and currently all of these trials present the same sounds.

    I’m looking to make the configuration a bit more subtle, but I’ve only ever constructed trial lists like this in Matlab so some questions that I had were:

    Is the set of newTrial elements made using PennController.Template() an array and if so, is it possible to use a template to construct my catch trials in a way similar to how it’s done for the main experiment trials (via rows in a CSV) and then push them into the running order from there? This would allow me to have a few different versions of the catch trials depending on the stimuli that the participant is assigned.

    Also, if the running order passed into modifyRunningOrder() is just an array of elements, would it be possible to, after pushing a catch trial, use something like array.slice() to get the previous “block” worth of trials, randomize it with randomize() and then use array.splice() to put them back into the main running order? This would have the effect of making the catch trial appear randomly in every block rather than at the end of every block.

    Let me know what you think, and if either would be more easily accomplished with the specific labeling method you mentioned in your previous reply.

    Thanks again for all of your help!

    Nickolas

    #5627
    Jeremy
    Keymaster

    Hi Nickolas,

    So if I understood you correctly, you need to present your trials in blocks of N trials each, each block consisting of N-1 test trials and 1 catch trial presented in a total random order, ie. within each block whether the catch trial occurs at the beginning, in the middle or at the end is random—as a consequence, no two catch trials can be separated by more than 2*(N-1) test trials. Did I get this right?

    I wrote this to do just what I described, but feel free to edit the code if your needs are different:

    function ShuffleInChunks(...args) {
        this.args = [];
        this.numbers = [];
        for (let i = 0; i < args.length; i++){
            if (i%2) this.numbers.push(args[i]);
            else this.args.push(args[i]);
        }
        
        this.run = function(arrays) {
            let newarray = [];
            for (let i = 0; i < arrays.length; i++) fisherYates(arrays[i]);
            while (arrays.filter(a=>a.length>0).length){
                let currentChunk = [];
                for (let i = 0; i < arrays.length; i++) {
                    let currentArray = arrays[i]
                    for (let n = 0; n < this.numbers[i]; n++){
                        if (currentArray.length) currentChunk.push(currentArray.pop());
                        else break;
                    }
                }
                fisherYates(currentChunk);
                newarray = newarray.concat(currentChunk);
            }
            return newarray;
        };
    }
    function shuffleInChunks(...args) { return new ShuffleInChunks(...args) }

    You can use it like this, which would give you blocks of 11 trials---10 test trials and 1 catch trial, randomly shuffled:

    Sequence( shuffleInChunks("test",10,"catch",1) )

    To answer your question, Template does not return an array, it actually feeds the native-Ibex items array with the output of each newTrial loop on the table's row. It is also executed asynchronously, which means that you cannot simply write a Template command in your script and operate on its output by just writing some more code below it. This is why I'm suggesting the solution above: not only is it conceptually cleaner (you can directly tell the trial order just by looking at Sequence) but the trial order as defined by Sequence is computed after the native-Ibex items array has been filled, including with the outputs of Template.

    Let me know if you have questions

    Jeremy

    #5641
    Ncomeau
    Participant

    Hi Jeremy,

    I added the code above to my script and it works well, thank you so much for the assist.
    While I was setting it up, I noticed a few things that I wanted to ask you about:

    Correct me if I’m wrong, but I think I remember seeing somewhere else on the forums that using a URL with “?withsquare=N” will set up your template with values corresponding to whatever rows are associated with Group = N+1 in the CSV files provided (eg withsquare=0 -> group 1). In my case, I have the pairings of main experiment audio files in one CSV (“main” trial template), and those for the catch trials in another (“catch” trial template). Calling ShuffleInChunks(“main”,14,”catch”,1) in my sequence to have 15 trial “blocks” works as intended. However, I noticed that for some reason every time I reopened the experiment to test, the files associated with the catch trial CSV were being preloaded just fine and the files for the main experiment trials were not. This seemed strange as all of the audio was supposed to be coming from the same successfully downloaded zip file in my AWS bucket. After making sure it wasn’t anything to do with my main.js code, I noticed a counter present under the counter section of my PCIbex experiment page and that it was at a number much higher than any of the group or withsquare numbers I would use. After deleting this, the preloading of all of the files works fine. It breaks after trying to reload the page without deleting it though. Any ideas on how to handle this? I don’t think it was ever an issue before adding this function in…

    The other thing I noticed is that so long as I have the modifyRunningOrder function uncommented, it appears to run (I’ve changed it’s functionality from inserting catch trials every 15 and breaks every 30 to just adding the breaks). Is that to be expected? I thought I would need to do something like modifyRunningOrder(ShuffleInChunks(“main”,14,”catch”,1)) in my sequence to have it add breaks properly. The intended effect is taking place, I’m just curious why it would be running without me explicitly calling it.

    Thanks again for the help, I really appreciate it!

    Nickolas

    #5643
    Jeremy
    Keymaster

    Hi Nickolas,

    1) I suspect that the audio filenames might not be consistent across groups in your main CSV table, for example maybe you typed uppercase extensions in some rows—you should double-check that
    The internal counter just keeps track of how many times your experiment was taken (since the last time you deleted the counter file, if you deleted it) but PennController (and Ibex, independently) cycle through it based on the number of groups in your design, eg. if you have 16 groups and a participant takes your experiment with counter 243, they’ll be assigned group 243%16=3

    2) You should not use ShuffleInChunks as the argument of modifyRunningOrder, in fact you should not call modifyRunningOrder: Ibex calls it internally. What you can do is (re)define it, and it will be executed after the whole sequence has been computed (from the Sequence command in PennController, from the shuffleSequence variable in native Ibex).
    PennController actually needs to define modifyRunningOrder, and this might cause conflicts with whatever you are trying to do. It’s better if you can avoid using the modifyRunningOrder method altogether. You can easily define your own function to insert an item ever N trials:

    function SepWithN(sep, main, n) {
        this.args = [sep,main];
    
        this.run = function(arrays) {
            assert(arrays.length == 2, "Wrong number of arguments (or bad argument) to SepWithN");
            assert(parseInt(n) > 0, "N must be a positive number");
            let sep = arrays[0];
            let main = arrays[1];
    
            if (main.length <= 1)
                return main
            else {
                let newArray = [];
                while (main.length){
                    for (let i = 0; i < n && main.length>0; i++)
                        newArray.push(main.shift());
                    for (let j = 0; j < sep.length; ++j)
                        newArray.push(sep[j]);
                }
                return newArray;
            }
        }
    }
    function sepWithN(sep, main, n) { return new SepWithN(sep, main, n); }

    Then, assuming you create a break trial labeled break, you can use it like this:

    Sequence(sepWithN( "break" , shuffleInChunks("main",14,"catch",1) , 30))

    Jeremy

    • This reply was modified 3 years, 3 months ago by Jeremy. Reason: replaced main.pop() with main.shift()
    #5645
    Ncomeau
    Participant

    Jeremy,

    The SepWithN function seems to work a lot better, so I’ve gone ahead and removed my changes to the definition of the modifyRunOrder function.

    Regarding the table / counter interaction, I am using my groups to assign one of eight mutually exclusive sets of stimuli (housed in different zip files on AWS) to participants so while the name differs across groups, I was not having any issue with preloading prior to setting up these functions to insert break/catch trials. Up until now, I’ve been forcing a particular group number by running my experiment with “?withsquare=0”. If my understanding is correct, this will load “subject_0.zip” from my AWS location, and the filenames in that zip file correspond to all of the lines of my CSV where the group field is 1. When the counter has been freshly deleted, the preloading of each individual file works as expected.

    Some things that I noticed today however were:
    For withsquare=1 (group->2), the debugger tells me that my subject_1.zip downloads fine, but there are not any lines in the debugger saying the individual files were successfully preloaded (and the experiment hangs on preloading). This is the same thing that happens when I don’t delete the counter for an attempt of withsquare=0. Is it possible that the counter is overriding my explicit selection of a group with withsquare?

    Another strange thing that I noticed is that rather than increasing upon every run of the experiment, my counter is increasing every trial. Is that due to sending results every trial perhaps? If so, I can just manually reset the counter to whatever its value was at the beginning of the script. The more that I think about it, doing my group assignment with counter rather than withsquare in the URL might be easier in the end as I would like to allow for the user to continue on to another set of data. This could be hard to do if they were initially assigned the last group with withsquare=7 and I would then somehow need to bring that back down to 0 again as opposed to just incrementing to 8.

    I hope this makes sense, and I appreciate your patience given that some of my design choices go against the grain for existing penncontroller functionality.

    Nickolas

    #5650
    Jeremy
    Keymaster

    Hi Nickolas,

    I think I’d need to see your experiment, would you mind sharing its URL with me?

    Jeremy

    #5653
    Ncomeau
    Participant

    Of course, I will send the link to the support email.

    #5669
    Ncomeau
    Participant

    Hi again Jeremy,

    I think there was an issue with sending my experiment links from my wisc.edu email or my gmail to support@pcibex.net and admin@pcibex.net. It said they were not deliverable due to being unverified addresses? I think we are going to handle group assignment with separate prolific experiments anyways now, but something I noticed about the withsquare URL parameter is that it works no matter what if I append “ExperimentName/server.py?withsquare=0”, but sometimes breaks if I only use “ExperimentName/experiment.html?withsquare=0” and the counter is not deleted beforehand.

    Some other miscellaneous questions I’ve had in finishing up the experiment are:
    Is it possible to append a tag to every line of the log on the final trial (and send_results function call) of my experiment? I am sending results on every trial in an effort to retain information should the participant quit early, but I still haven’t found a great way to indicate the relevant final chunk of logged items in my results file.

    Also, is there a way to make the progress bar larger / move its position on screen, and adjust or remove the “Sending results to server…” text?

    #5670
    Jeremy
    Keymaster

    Hi Nickolas,

    Adding ?withsquare=N to experiment.html will not override the internal counter, only server.py is able to handle this, so your experiment probably breaks because of a mismatch between the value of the internal counter and whatever you do with withsquare in your script (eg. retrieving it using GetURLParameter). If you need to use withsquare, always use it with server.py.

    Could you use .log to add a tag (effectively, a value in an extra column) to the lines of your last trial? Alternatively you could also give it a unique label, e.g. “final.” Or note down its ID number by taking the experiment yourself once and use it in your analyses.

    The CSS of the progress bar is defined in global_main.css—if you can’t directly modify this file, you should be able to upload a file named global_z.css and override the rules (use !important if necessary).

    You can define the value of the variable sendingResultsMessage to modify the corresponding text, as documented here

    Jeremy

    • This reply was modified 3 years, 9 months ago by Jeremy.
    #5753
    Ncomeau
    Participant

    Hi Jeremy,

    Going along with the question I had above regarding changing the sendingResultsMessage: Is there a way to change/remove the text that appears during a checkpreload function call? It would be like changing things from “Please wait while the resources are preloading. This may take up to 30s.” to “Loading, please wait”. A little nit-picky but I mostly want to avoid jargon / participants feeling like they missed something.

    Thanks again,

    Nickolas

    #5754
    Jeremy
    Keymaster

    Hi Nickolas,

    I haven’t added an option to customize this message yet, so you’ll have to use the hack from this message

    const replacePreloadingMessage = ()=>{
        const preloadingMessage = $(".PennController-PennController > div");
        if (preloadingMessage.length > 0 && preloadingMessage[0].innerHTML.match(/^<p>Please wait while the resources are preloading/))
            preloadingMessage.html("<p>Loading, please wait</p>");
        window.requestAnimationFrame( replacePreloadingMessage );
    };
    window.requestAnimationFrame( replacePreloadingMessage );

    Jeremy

Viewing 15 posts - 1 through 15 (of 15 total)
  • You must be logged in to reply to this topic.