Randomizing trials, with 1/5 of trials followed by a comprehension question

PennController for IBEX Forums Support Randomizing trials, with 1/5 of trials followed by a comprehension question

Viewing 9 posts - 1 through 9 (of 9 total)
  • Author
    Posts
  • #9547
    jackying
    Participant

    Hi Jeremy! I’m working on programming an experiment which asks participants to choose a word between two options for a blank in each conversation. Here’s the demonstration link. https://farm.pcibex.net/r/hQtoVE/

    My first question is on presenting conversations. I’m using a .csv file that has a column for conversations, and for each conversation, I use <br> at the end of each speaker’s line to separate the lines, so that the conversations look like what’s in the demonstration link (i.e., <br> doesn’t show during presentation of conversations). But for some conversations, all the lines are connected by <br> in the text, and the lines are not separated at all. I tried checking texts in the .csv file but didn’t notice any differences between the conversations that work and ones that don’t. Is there something I can do to make sure all conversations show properly in separate lines?

    My second question is on randomization. There are in total 60 conversations/trials for each participant. I want the order of the 60 conversations/trials randomized, and I want 1/5 of the trials to be followed by a comprehension question. What I did was that I included a column for comprehension questions in my .csv file (the input file), so that all conversations come with a comprehension question. I wrote two templates for the trials, one named “trial” (participants simply fill in the blank for a given conversation) and the other “trial-Q” (participants fill in the blank for a conversation and then answer a question). The difference between the two is basically that “trial-Q” prints the comprehension question and the yes/no options, but “trial” doesn’t. Below is what I wrote for the sequence, with reference to a post where you suggested codes for “subsequence”. https://www.pcibex.net/forums/topic/partial-randomization-of-trial-sequence/

    Sequence("intro", "practice", subsequence(repeat(randomize("trial"),4),"trial-Q"),"send", "end")

    I think the part subsequence(repeat(randomize("trial"),4),"trial-Q”) does something like, I randomly draw 5 trials from the 60 trials, and for these 5 trials, the first 4 trials are assigned to the “trial” template (with no comprehension question) and are randomized for order, and the last trial is assigned to the “trial-Q” template (with a comprehension question). If I do this for all 60 trials, I get one “trial-Q” out of every 5 trials, so 1/5 of the trials are followed by a comprehension question.

    I have two concerns here, which I don’t know how to address.
    One is that with my codes, some conversations might repeat if they happen to be drawn for a second time. Is there something I can do in the codes to avoid this?

    Another concern I have is that participants might get the pattern of when a “trial-Q” is gonna show up — it always happens after 4 “trial” templates, and I certainly don’t want this. Is there a way these “trial-Q”s can be randomly distributed/interleaved in the total of 60 trials?

    #9552
    Jeremy
    Keymaster

    Hi,

    1. Starting with line 316 of Context_5y_mot_1.csv, some <br>s are replaced with [br]s

    2. Each Template command browses the whole CSV file and outputs as many trials labeled accordingly. So your script outputs 120 trials, half labeled “trial” and half labeled “trial-Q”, which are duplicates of each other except for the addition of a comprehension question for the latter.

    I think the part subsequence(repeat(randomize("trial"),4),"trial-Q") does something like, I randomly draw 5 trials from the 60 trials, and for these 5 trials, the first 4 trials are assigned to the “trial” template (with no comprehension question) and are randomized for order, and the last trial is assigned to the “trial-Q” template (with a comprehension question).

    This is not how Template and trial references work. Each trial has a label, and you can refer to those labels in setting up the experiment’s sequence. Simply referencing "trial-Q" will refer to all the trials labeled “trial-Q” (60 trials in your case); randomize("trial") represents all the trials labeled “trial” (again, 60 trials in your case) but in a randomized order. repeat(randomize("trial"),4) represents a way of iterating over series of four trials from that set of 60 trials; inserting it inside subsequence will iterate until the series exhausts all the trials from the set. So subsequence(repeat(randomize("trial"),4),"trial-Q") will repeat a series of 4 (different) trials labeled “trial” followed by a (different) trial from the set of the 60 trials labeled “trial-Q”, until all the former have been drawn: in the end, you’ll have 15 subsequences of a series of 4 “trial” trials followed by one “trial-Q” trial, so all the 60 “trial” trials, and 15 “trial-Q” trials. Note that those 15 “trial-Q” trials will have been generated from the same CSV lines as 15 of the “trial” trials, so you’ll have duplicates, in that sense

    If you know in advance you’ll have 60 trials, the easiest thing to do is create an array of 60 boolean entries, 15 true and 45 false, and shuffle that array. Then, you only generate 60 trials all labeled the same way, and pop the value at the top of the array for each trial: if it’s true then you include the question, otherwise you don’t.

    arrayTrialQ = [...Array(60)].map((v,i)=>i<15)
    fisherYates(arrayTrialQ)
    
    Template("Context_5y_mot_1.csv" , row =>
      newTrial("trial",
        newVar("correct").global()
        ,
        newTimer("pre-trial", 500).start().wait()
        ,
        newText("target1", row.context).center().print()
        ,
        newText("line1", " ").center().print()
        ,
        newScale("answer", row.D1, row.D2) 
          .center()
          .button()
          .print()
          .log()
          .wait()
        ,
        getScale("answer")
          .test.selected(row.ans==row.D1 ? row.D1 : row.D2)
          .success( getVar("correct").set(true) )
          .failure( getVar("correct").set(false) )
          .remove()
        ,
        newTimer("post-trial", 500).start().wait()
        ,
        getText("target1").remove()
        ,
        getText("line1").remove()
        ,
        newVar("correctQ").global().set("NA")
        ,
        ...( arrayTrialQ.pop() == true ? [
          newText("target2", row.question)
            .center().print()
          ,
          newText("line2", " ")
            .center().print()
          ,
          newScale("answer-Q", "[Yes]", "[No]") 
            .center()
            .button()
            .print()
            .log()
            .wait()
          ,
          getScale("answer-Q")
            .test.selected(row.ansQ=='[Yes]' ? "[Yes]" : "[No]")
            .success( 
                getVar("correctQ").set(true),
            )
            .failure( 
                getVar("correctQ").set(false),
            )
          ,
          newTimer("post-Q", 500).start().wait()
        ] : [] )
      )
      .log("context", row.context)
      .log("Correct", getVar("correct"))
      .log("Question", getVar("correctQ"))
    )

    Then just use randomize("trial") in Sequence instead of the whole subsequence command. In the results file, the “Question” column will read “NA” for those trials without a comprehension question, and “true” or “false” for the ones with a comprehension question

    Jeremy

    #9567
    jackying
    Participant

    Hi Jeremy! Thank you so much for the help!

    #9577
    jackying
    Participant

    Hi Jeremy, the codes you suggested work! Thanks!

    Here’s two follow-up questions, and they’re still about the same experiment asking participants to choose a word between two options for the blank in each conversation. Here’s link to the experiment with revised codes. https://farm.pcibex.net/r/hQtoVE/

    1. How can I have the 60 test items/conversations randomly drawn from a master list (.csv file) with 90 blanked conversations?
    PS: I did found a relevant post https://www.pcibex.net/forums/topic/choosing-subet-of-items-to-present/, but my case seems to be more complicated.

    These 90 conversations are labeled by the answer to the blank in each conversation and fall into 6 categories (the number of items for each category is put in parentheses and they add up to 90), let’s say, A (20), B (20), C (15), D (15), E (10), and F (10). First, I randomly take out 5 items from each category (i.e., a total of 30 items taken out or eliminated), so I have the remaining 60 items to start with as the actual set of test items.

    2. For the items/conversations in each category after elimination (suggested above), how do I assign one option pair to some conversations in that category and another option pair to the other conversations in the same category?

    Following the idea you suggested with using Boolean entries and given my case/needs, I’m considering to assign 5 “false”s to items in each category and the rest of the items will be “true”. The idea is basically that I want to give conversations different option pairs depending on 1) the category of (i.e., answer to) the conversation and 2) the “true”/”false” value assigned. So what makes this new version different from the old version (the demonstration link attached above) is that, while I already have separate columns (i.e., D1 and D2 in the .csv file) for option pairs in the old version, in the new version, there will not be predetermined option columns for each conversation — instead, the codes will decide what option pairs to be assigned to each conversation depending on the category and the “true”/”false” value assigned in each case.

    For instance, for items in categories A and B, if they are assigned “true”, the option pair for the blank will be A and B. For items assigned “false” in A, the option pair will be A and C, and for those “false” items in B, the option pair will be B and D. Is there a way for me to achieve this? I think I just need example codes for cases with “true” and “false” in category A, and I should be able to extend that to other categories with relevant modifications. Moreover, is it possible for me to log the option pairs assigned to each conversation (for the exact thing to be logged, maybe something more general like “true” or “false”, or it could be something more detailed like “A/B” for “true” items in category A)?

    Some more trivial questions include:
    a) Because I need to assign different option pairs to the six categories (A to F), do I need more than one template to make that happen? (Maybe not if the assignment of option pair should be done separately before the use of template or templates.)
    b) Does the assignment of option pairs affect the code I should be using for the line Sequence("intro", "practice", randomize("trial"),"send", "end")?

    I hope what I want to achieve is clearly stated above, but feel free to let me know if there’s places I didn’t explain well!
    Thank you so much for your time!

    #9580
    Jeremy
    Keymaster

    1. I imagine you’ll have the 90 conversations listed in the CSV table, and that you’ll have a column that indicates which category each conversation falls into. I’ll assume you name that column category with the possible values A, B, C, D, E and F.

    I’m not sure what you mean by “taken out”/”eliminated”: should those 30 items just never appear at all, in any form, in your experiment? Or should they still be included in the experiment, but just without a comprehension question, while the other 60 trials will indeed have a comprehension question? The solutions are significantly different depending on which of these two options you mean to implement

    If it’s the former, then you just need to add a spin to the earlier method: instead of using a single array of 60 entries with 12 trues vs 48 falses, you’ll want to use 6 arrays (one per category) each with 5 falses and the rest of entries set to true. Then you can use the category column in newTrial to look up the appropriate array:

    arraysTrialQ = {
      A: [...Array(20)].map((v,i)=>i>4),
      B: [...Array(20)].map((v,i)=>i>4),
      C: [...Array(15)].map((v,i)=>i>4),
      D: [...Array(15)].map((v,i)=>i>4),
      E: [...Array(10)].map((v,i)=>i>4),
      F: [...Array(10)].map((v,i)=>i>4),
    }
    fisherYates(arraysTrialQ.A)
    fisherYates(arraysTrialQ.B)
    fisherYates(arraysTrialQ.C)
    fisherYates(arraysTrialQ.D)
    fisherYates(arraysTrialQ.E)
    fisherYates(arraysTrialQ.F)
    
    Template("ContextBank.csv" , row =>
      newTrial("trial",
        // ...
        ...(arraysTrialQ[row.category].pop() == true? [
          newText("target2", row.question)
          // ...
    

    If, on the other hand, you want to discard them altogether, things are quite different. You could insert the value from the category column in the trial’s label, so you have the ability to discriminate between them when controlling your sequence (ie. newTrial("trial-"+row.context,). Then you could then use the custom function pick to draw the desired number of trials from each category:

    aTrials = randomize("trial-A")
    bTrials = randomize("trial-B")
    cTrials = randomize("trial-C")
    dTrials = randomize("trial-D")
    eTrials = randomize("trial-E")
    fTrials = randomize("trial-F")
    
    Sequence(
      "intro", "practice", 
      pick(aTrials,15), // 15 out of 20
      pick(bTrials,15), // 15 out of 20
      pick(cTrials,10), // 10 out of 15
      pick(dTrials,10), // 10 out of 15
      pick(eTrials, 5), //  5 out of 10
      pick(fTrials, 5), //  5 out of 10
      "send", "end"
    )

    2. Note that if you assign 5 falses to the E and F trials, you will effectively end up with all of them being assigned false, as you can easily see from the piece of code I give above—maybe you’re fine with that, but I wanted to bring your attention to it

    You can indeed use a method similar to the one above, using arrays of trues and falses.

    optionsCategories = {
      A: ["option1a","option2a","option3a"],
      B: ["option1b","option2b","option3b"],
      C: ["option1c","option2c","option3c"],
      D: ["option1d","option2d","option3d"],
      E: ["option1e","option2e","option3e"],
      F: ["option1f","option2f","option3f"]
    }
    arraysTrialOptions = {
      A: [...Array(15)].map((v,i)=>i>4),
      B: [...Array(15)].map((v,i)=>i>4),
      C: [...Array(10)].map((v,i)=>i>4),
      D: [...Array(10)].map((v,i)=>i>4),
      E: [...Array(5)].map((v,i)=>i>4),
      F: [...Array(5)].map((v,i)=>i>4),
    }
    fisherYates(arraysTrialOptions.A)
    fisherYates(arraysTrialOptions.B)
    fisherYates(arraysTrialOptions.C)
    fisherYates(arraysTrialOptions.D)
    fisherYates(arraysTrialOptions.E)
    fisherYates(arraysTrialOptions.F)
    
    Template("ContextBank.csv" , row =>
      newTrial("trial-"+row.category,
        // ...
        newVar("trueFalseOption").global().set( v=>arraysTrialOptions[row.category].pop() )
        ,
        newScale("answer", "answer1", "answer2")
          .label(0, optionsCategories[row.category][0] ) // always use the same first answer
          .label(1, (optionsCategories[row.category][trueFalseOption==true ? 1 : 2]) )
          .center()
          .button()
          // ...
      )
      .log("trueFalseOption", getVar("trueFalseOption"))
    )
    

    Re. questions a/b, I think the codes in this post sort of answer them already but I invite you to read the documentation about how labels and Template work, and this tutorial page on using a table. As you can see, I cannot give one general answer to these questions, as it all depends on how specifically you want to address the other questions, but you can answer them once you know how generating trials from a table and referencing them using their label in Sequence works

    Jeremy

    #9635
    mwf2120
    Participant

    Hi! I have a related-ish question. I am partially randomizing the color and order of 4 images on a canvas. After initial randomization, I’d like for the participant to see the same color and order repeatedly, for each practice trial and experimental trial. Is there a way to, after randomization, make a canvas somewhat of a “global variable” that can be called in a later trial?

    I am randomizing 4 pics into one of 6 groups with this:

    function createColorGroup(int) {
        var bugba = ["blue", "green", "black", "bugba"];
        var gbuba = ["green", "blue", "black", "gbuba"];
        var bagbu = ["black", "green", "blue", "bagbu"];
        var gbabu = ["green", "black", "blue", "gbabu"];
        var babug = ["black", "blue", "green", "babug"];
        var bubag = ["blue", "black", "green", "bubag"];
        
        var groups = [bugba, gbuba, bagbu, gbabu, babug, bubag];
        
        return groups[int]; 
    }
    
    // color order var generation
    newTrial("color_order",
        newVar("colorOrder")
            .global()
            .set(color_order_label)
    ).log("id", getVar("subjID"))
    .log("color_group", color_order_label)

    Then, I am getting images onto the canvas like this:

    newImage(getImageEmotion(row.leftImage)+"", leftColor + row.leftImage +"")
            .cssContainer("border", "solid 2px black")
            .cssContainer("background-color", "white")
            //.print()
        ,
        newImage("calmImage", centerColor + "calmnolabel.jpg")
            .cssContainer("border", "solid 2px black")
            .cssContainer("background-color", "white")
            //.print()
        ,
        newImage(getImageEmotion(row.rightImage)+"", rightColor + row.rightImage +"")
            .cssContainer("border", "solid 2px black")
            .cssContainer("background-color", "white")
            //.print()
        ,
        newCanvas("labels", 600, 200)
            .add("center at 18%", "middle at 50%", getImage(getImageEmotion(row.leftImage)+""))
            .add("center at 50%", "middle at 50%", getImage("calmImage"))
            .add("center at 82%", "middle at 50%", getImage(getImageEmotion(row.rightImage)+""))
            .color("yellow")
            .cssContainer("border", "solid 2px black")
            .settings.center()
            .print()

    Then, I’d like to do something like this:

    Template(
        GetTable("emoMusicGroupedPracticeV2.csv")
        ,
        row => newTrial("PracticeEmoMusic",
            newAudio(row.audiofile)
                .center()
                .print()
                .wait() 
            ,
            newText("Q", "<p>Whose face matches the music?</p>")
                .center()
                .css("font-size", "1.5em")
                .print()
            ,
            newSelector("eugenies")
                .once()
                .frame("dashed 0px white")
            ,
            getCanvas("labels")

    but am running into trouble. Thanks for your help!!

    #9641
    Jeremy
    Keymaster

    Hi,

    Some elements are missing from the pieces of code you posted (how color_order_label, leftColor, centerColor, rightColor and getImageEmotion are defined, where/whether createColorGroup is called elsewhere) but if I understand your question correctly, you should be able to do something like this:

    function createColorGroup(int) {
        var bugba = ["blue", "green", "black", "bugba"];
        var gbuba = ["green", "blue", "black", "gbuba"];
        var bagbu = ["black", "green", "blue", "bagbu"];
        var gbabu = ["green", "black", "blue", "gbabu"];
        var babug = ["black", "blue", "green", "babug"];
        var bubag = ["blue", "black", "green", "bubag"];
        
        var groups = [bugba, gbuba, bagbu, gbabu, babug, bubag];
        
        return groups[int]; 
    }
    color_group = createColorGroup(Math.round(6*Math.random()))
    leftColor = color_group[0]
    centerColor = color_group[1]
    rightColor = color_group[2]
    color_order_label = color_group[3]
    
    Template( "emoMusicGroupedPracticeV2.csv" ,
      row => newTrial("PracticeEmoMusic",
        newAudio(row.audiofile).center().print().wait() 
        ,
        newText("Q", "<p>Whose face matches the music?</p>").center().css("font-size", "1.5em").print()
        ,
        newSelector("eugenies").once().frame("dashed 0px white")
        ,
        defaultImage.cssContainer({border:"solid 2px black","background-color":"white"})
        ,
        newImage(String(getImageEmotion(row.leftImage)), leftColor+row.leftImage),
        newImage("calmImage", centerColor + "calmnolabel.jpg"),
        newImage(String(getImageEmotion(row.rightImage)), rightColor+row.rightImage)
        ,
        newCanvas("labels", 600, 200)
          .add("center at 18%", "middle at 50%", getImage(String(getImageEmotion(row.leftImage))))
          .add("center at 50%", "middle at 50%", getImage("calmImage"))
          .add("center at 82%", "middle at 50%", getImage(String(getImageEmotion(row.rightImage))))
          .color("yellow")
          .cssContainer("border", "solid 2px black")
          .center()
          .print()
        // etc.
      )
      .log("id", getVar("subjID"))
      .log("color_group", color_order_label)
    )

    Jeremy

    #9706
    jackying
    Participant

    Hi Jeremy,

    Thank you so much for your help!

    I’ve figured out codes to run the experiment, and I have three more questions with presentation and formatting. Here’s the demonstration link https://farm.pcibex.net/r/ogzxLk/

    1) I want to present the conversations in a way that one line is added to the conversation with each click/keypress. That is,

    First screen:
    line 1

    second screen:
    line 1
    line 2

    third screen:
    line 1
    line 2
    line 3

    etc

    I used <br> to segment lines in each conversation, and I’m wondering if there’s a simple way for me to implement the abovementioned way of presentation with my use of <br>(s).

    2) I tried to find in the PCibex manual codes to adjust spaces between lines but didn’t find them. It would be nice if you can suggest ways to adjust the default spaces between lines so that they don’t look too cramped.

    3) I want to present aligned utterances of different speakers so that it’s easier to see who’s talking. Currently, my texts look like this.

    Mother: xxxxx.
    xxxxx.
    Child: xxxxx.
    Mother: xxxxxx.

    So the format I want would be something like the following.

    Mother:   xxxxxxx.
              xxxxxxx.
    Child:    xxxxxxx.
    Mother:   xxxxxxx.

    My input files are .csv files (originally .xlsx files), so I’m guessing I’ll either have to edit the texts in excel or do it over PCibex (if there’s a way to do that).

    These are my remaining questions. Thank you for your time!

    • This reply was modified 1 year, 6 months ago by jackying.
    #9708
    Jeremy
    Keymaster

    Hi,

    1+3) You’ll need to split row.context at each <br> so you get a list of each line separately, so you can print one line on the page then use a Key element to wait for a keypress, then print the next line, wait for a keypress again, etc. If you want an alignment like you describe, you’ll need to use a table HTML element, which is not directly implemented as a PennController element, so you’ll need to use javascript to inject it in the page:

    newKey("next", " ")
    ,
    newText("context", "
    ").print() , ...row.context.split("
    ").map( line => [ newFunction(()=>{ const tableRow = document.createElement("tr"); const speaker = document.createElement("td"); const speech = document.createElement("td"); speaker.innerText = line.replace(/^([^:]+:).+$/,"$1").replace(/\n/gi,''); speech.innerText = line.replace(/^[^:]+:/,"").replace(/\n/gi,''); tableRow.append(speaker); tableRow.append(speech); document.querySelector("#context").append(tableRow); }).call() , getKey("next").wait() ] ).flat()

    2) You can add custom CSS to control visual rendering. In the case of the piece of code above, I’d suggest you target the table with something like this in global_z.css:

    table#context tr {
      padding-bottom: 0.5em;
    }
    table#context tr td {
      padding-right: 0.5em;
    }
    

    Jeremy

Viewing 9 posts - 1 through 9 (of 9 total)
  • You must be logged in to reply to this topic.