Question about assigning conditions, groups + randomization

[KateK]

Hi all,

I am currently building an experiment and would love some guidance as to how to implement the desired randomization scheme.

We want participants to be assigned to one condition in each “pair” of adjectives, with 11 “pairs” existing in total. Then, we want the participants to see three different constructions associated with each specific conditionXpair, but we want the three constructions to be shuffled around so that no one encounters all three constructions for a particular conditionXpair back-to-back.

Additionally, each condition of the “pair” has a polarity assigned (POS vs. NEG) and we want participants to be assigned an equal number of POS and NEG polarity conditionXpairs.

The resources tab for the draft experiment build contains a mock-up of the current csv.

https://farm.pcibex.net/r/Zhoqih/

Thank you in advance for your insight!

[Jeremy]

Hi,

There are many ways to go about this, but here is a suggestion. You can first use Template to read the CSV table into a javascript dictionary, then manipulate that dictionary in a later Template command to create three items per row, but only in the NEG or POS condition for each pair. Here’s on way of doing it:

// Note: Template gives us access to the content of the CSV file, but it executes the function it is passed
// only after this whole script file has been executed, this is why we need to place our code inside Template

const targets = {} // store the rows from the CSV file into a javascript dictionary
Template("target_test.csv", row => {
    // the dictionary lists all the same-pair items under the same key
    if (targets[row.pair]===undefined) targets[row.pair] = {POS:[],NEG:[]};
    targets[row.pair][row.target_polarity].push(row);
    return {}; // we're not actually generating trials here: return an empty object
})

AddTable("dummy", "a,b\rc,d") // Dummy one-row table to execute some code from Template once after the code above
Template("dummy", () => {
    const targetKeys = Object.keys(targets);
    fisherYates(targetKeys);   // shuffle the references to the pairs
    let new_targets = []; // this will contain half the items (only POS or only NEG for each pair)
    for (let i=0; i<targetKeys.length; i++) // keep the POS rows for the first half, the NEG rows for the second half
        new_targets.push( ...targets[targetKeys[i]][ i<targetKeys.length/2 ? "POS" : "NEG"] );
    // Create three items per row that we kept
    new_targets = new_targets.map(t=>
        [ [t['construction 1'],t['c1 question']],[t['construction 2'],t['c2 question']],[t['construction 3'],t['c3 question']] ].map(t2=>
            ["experiment_"+t.pair,"PennController",newTrial(
                newText([t.context_adj,t.target_adj,t.target_polarity,t2[0],t2[1]].join("<br>")).print()
                ,
                newButton("Next").print().wait()
            )]
        ) // this map returns an array of 3 trials
    ).flat(); // flatten the array to have all the trials at the root, instead of having a series of arrays of 3 trials
    // Shuffle new_targets as long as we can find three items in a row that come from the same pair
    while (new_targets.find( (v,i)=>i<(new_targets.length-2) && v[0].split('_')[1]==new_targets[i+1][0].split('_')[1] && v[0].split('_')[1]==new_targets[i+2][0].split('_')[1] ))
        fisherYates(new_targets);
    window.items = new_targets; // now add the trials to the experiment's items
    return {}; // we added the items manually above: return an empty object from Template
})

Jeremy

[KateK]

Hi Jeremy,

I tried to place this code in my experiment and when I try to run it, it freezes and crashes the page. Do you have any advice?

Best,
Kate

[Jeremy]

Hi Kate,

This is because your “pair” column has a trailing space character at the end, which makes all the references to the pair column in the code return undefined. As a result, every single trial generated by it is labeled "experiment_undefined" and therefore the while loop does find three successive trials in new_targets that share a label forever no matter how many times they are shuffled (because they are all labeled "experiment_undefined") and so the script gets stuck there. Just delete the trailing space and it should work

Jeremy

[KateK]

Hi Jeremy,

Thank you again for your help with this!

It doesn’t seem to be printing things from the mock-up test template or producing the items and presenting them – do you think you could help me figure out what went awry? Is it an issue with the column names between the test and filler items?

Here is the experiment as it exists now: https://farm.pcibex.net/r/Zhoqih/

I appreciate your help!

Kate

[Jeremy]

Hi Kate,

You’re not actually inserting the items in your experiment. The items created by the code above are labeled after the format “experiment_*”. If you replace "experiment", by startsWith("experiment"), in your Sequence command then you’ll see the items at the end of your sequence

Jeremy

[KateK]

This is great! Thanks so much for catching that.

With my filler items, I have a context sentence appear first, then a follow up question with a Likert scale. Do you know how I can apply this to the items created by the code above?

[KateK]

A follow-up – I tried to edit the code and inserted my .csv with the actual items in it and unfortunately can’t seem to get it to run. Could you take a look: https://farm.pcibex.net/r/Zhoqih/?

[Jeremy]

The debugger reports this error: TypeError: targets[row.pair][row.target_polarity] is undefined

Your CSV file does not contain the same values in the target_polarity column as in the former CSV file. The former CSV file had POS and NEG, hence the recurrent references to POS and NEG in the code. In this file, you have posi, neg and NA in that column. I would suggest you replace the occurrences of POS and NEG with posi and neg, respectively, but you’ll still be left with unhandled NAs (which also affects the “pair” column by the way). It seems to me that those NA rows are not of the same nature as the posi and neg ones, in that they won’t fall under the same crossing/distribution of conditions that you describe in your initial post, and that they should therefore live in a separate CSV file referenced somewhere else in your code

Regarding your other question, just edit the newTrial in the code to generate trials that suit you

Jeremy

[KateK]

Jeremy,

Thank you so much! I removed the NA rows for now and will give them a new home in another .csv!

Do you know what I should do in order to randomize the presentation of fillers and the new items generated from the Javascript dictionary?

Here is my most recent build: https://farm.pcibex.net/r/KTUJvz/.

[KateK]

Hi Jeremy,

Sorry to bug you again – should I/can I use the same Javascript dictionary (aka, duplicate) to generate and randomize three trials from the N/A items? I just put them in a separate spreadsheet and am not sure where to go with it.

Best,
Kate

[Jeremy]

Hi,

Do you know what I should do in order to randomize the presentation of fillers and the new items generated from the Javascript dictionary?

A common solution is to use rshuffle, eg rshuffle("experiment",startsWith("experiment_"))

should I/can I use the same Javascript dictionary (aka, duplicate) to generate and randomize three trials from the N/A items?

You cannot use the part that handles polarity, since your NA rows won’t fit there. Unlike the non-NA target rows, for which you only want to keep half of them (either the posi ones or the neg ones) it seems that you’re OK with generating three trials for every single NA row. That wouldn’t require any special code, as long as you list one trial per row instead of combining three in a single row, eg:

Table sample

adjective number,adjective class,pair,context_adjective,target_adjective,target_polarity,subject property,contextsetter,targetsentence,construction
1,minimum_partial,NA,healthy,sick,NA,inanimate,I know that the apple tree and the pear tree are both healthy.,Which one is sicker than the other?,comparative
1,minimum_partial,NA,healthy,sick,NA,inanimate,I know that the tomato plant and the raspberry bush  are both healthy.,Which plant is as sick as these two?,equative
1,minimum_partial,NA,healthy,sick,NA,inanimate,I know that the orchid and the cactus are both healthy.,How sick are they?,question
2,minimum_partial,NA,dry,wet,NA,inanimate,I know that the blue towel and the green towel are both dry.,Which one is wetter than the other?,comparative
2,minimum_partial,NA,dry,wet,NA,inanimate,I know that the white mop and the yellow mop are both dry.,Which mop is as wet as these two?,equative
2,minimum_partial,NA,dry,wet,NA,inanimate,I know that Patrick's backpack and his hat are both dry.,How wet are they?,question

Code

Template("minimumstandard.csv", myCustomTrialFunction)

Jeremy

[KateK]

Hi Jeremy!

Thanks, this helped a lot.

I’m unfortunately still having some trouble getting the aesthetics of my Experiment-Target items (generated by the .js dictionary at the top of the script) to match my Experiment-Filler items. I want the Target item optics to match the setup of the Filler items… I can’t figure out how to put a spacer between the Context Sentence and Question in the new_targets like I have for the other items.

I’m also having some weird issues with making sure all the item text is centered; I’m not sure why some of them will center and others won’t.

You can see the script here: https://farm.pcibex.net/r/jvWllE/

Any advice would be appreciated!

Kate

[Jeremy]

Hi Kate,

You can use the same function to generate all your trials:

Template("dummy", () => {
    const targetKeys = Object.keys(targets);
    fisherYates(targetKeys);   // shuffle the references to the pairs
    let new_targets = []; // this will contain half the items (only POS or only NEG for each pair)
    for (let i=0; i<targetKeys.length; i++) // keep the POS rows for the first half, the NEG rows for the second half
        new_targets.push( ...targets[targetKeys[i]][ i<targetKeys.length/2 ? "posi" : "neg"] );
    // Create three items per row that we kept
    new_targets = new_targets.map(t=>
        [ {contextsetter: t['contextcomparative'], contextquestion: t['comparativequestion']},
          {contextsetter: t['contextequative'], contextquestion: t['equativequestion']},
          {contextsetter: t['contextquestion'], contextquestion: t['questionquestion']}
        ].map(row => ["experiment_"+t.pair,"PennController", myCustomTrialFunction(row)] ) // this map returns an array of 3 trials
    ).flat(); // flatten the array to have all the trials at the root, instead of having a series of arrays of 3 trials
    // Shuffle new_targets as long as we can find three items in a row that come from the same pair
    while (new_targets.find( (v,i)=>i<(new_targets.length-2) && v[0].split('_')[1]==new_targets[i+1][0].split('_')[1] && v[0].split('_')[1]==new_targets[i+2][0].split('_')[1] ))
        fisherYates(new_targets);
    window.items = new_targets; // now add the trials to the experiment's items
    return {}; // we added the items manually above: return an empty object from Template
})

This way you necessarily get the same rendering for all your trials

Jeremy

[KateK]

Hi Jeremy,

Thanks! Unfortunately, it seems like the target item’s questions are not printing with this new code inserted. I can’t tell why the questions are printing for fillers and not targets, even though the aesthetics have been fixed. Could you help me figure out why?

https://farm.pcibex.net/r/lYFvPD/

I appreciate it!

[Author]

Posts