[Jeremy]

1. I imagine you’ll have the 90 conversations listed in the CSV table, and that you’ll have a column that indicates which category each conversation falls into. I’ll assume you name that column category with the possible values A, B, C, D, E and F.

I’m not sure what you mean by “taken out”/”eliminated”: should those 30 items just never appear at all, in any form, in your experiment? Or should they still be included in the experiment, but just without a comprehension question, while the other 60 trials will indeed have a comprehension question? The solutions are significantly different depending on which of these two options you mean to implement

If it’s the former, then you just need to add a spin to the earlier method: instead of using a single array of 60 entries with 12 trues vs 48 falses, you’ll want to use 6 arrays (one per category) each with 5 falses and the rest of entries set to true. Then you can use the category column in newTrial to look up the appropriate array:

arraysTrialQ = {
  A: [...Array(20)].map((v,i)=>i>4),
  B: [...Array(20)].map((v,i)=>i>4),
  C: [...Array(15)].map((v,i)=>i>4),
  D: [...Array(15)].map((v,i)=>i>4),
  E: [...Array(10)].map((v,i)=>i>4),
  F: [...Array(10)].map((v,i)=>i>4),
}
fisherYates(arraysTrialQ.A)
fisherYates(arraysTrialQ.B)
fisherYates(arraysTrialQ.C)
fisherYates(arraysTrialQ.D)
fisherYates(arraysTrialQ.E)
fisherYates(arraysTrialQ.F)

Template("ContextBank.csv" , row =>
  newTrial("trial",
    // ...
    ...(arraysTrialQ[row.category].pop() == true? [
      newText("target2", row.question)
      // ...

If, on the other hand, you want to discard them altogether, things are quite different. You could insert the value from the category column in the trial’s label, so you have the ability to discriminate between them when controlling your sequence (ie. newTrial("trial-"+row.context,). Then you could then use the custom function pick to draw the desired number of trials from each category:

aTrials = randomize("trial-A")
bTrials = randomize("trial-B")
cTrials = randomize("trial-C")
dTrials = randomize("trial-D")
eTrials = randomize("trial-E")
fTrials = randomize("trial-F")

Sequence(
  "intro", "practice", 
  pick(aTrials,15), // 15 out of 20
  pick(bTrials,15), // 15 out of 20
  pick(cTrials,10), // 10 out of 15
  pick(dTrials,10), // 10 out of 15
  pick(eTrials, 5), //  5 out of 10
  pick(fTrials, 5), //  5 out of 10
  "send", "end"
)

2. Note that if you assign 5 falses to the E and F trials, you will effectively end up with all of them being assigned false, as you can easily see from the piece of code I give above—maybe you’re fine with that, but I wanted to bring your attention to it

You can indeed use a method similar to the one above, using arrays of trues and falses.

optionsCategories = {
  A: ["option1a","option2a","option3a"],
  B: ["option1b","option2b","option3b"],
  C: ["option1c","option2c","option3c"],
  D: ["option1d","option2d","option3d"],
  E: ["option1e","option2e","option3e"],
  F: ["option1f","option2f","option3f"]
}
arraysTrialOptions = {
  A: [...Array(15)].map((v,i)=>i>4),
  B: [...Array(15)].map((v,i)=>i>4),
  C: [...Array(10)].map((v,i)=>i>4),
  D: [...Array(10)].map((v,i)=>i>4),
  E: [...Array(5)].map((v,i)=>i>4),
  F: [...Array(5)].map((v,i)=>i>4),
}
fisherYates(arraysTrialOptions.A)
fisherYates(arraysTrialOptions.B)
fisherYates(arraysTrialOptions.C)
fisherYates(arraysTrialOptions.D)
fisherYates(arraysTrialOptions.E)
fisherYates(arraysTrialOptions.F)

Template("ContextBank.csv" , row =>
  newTrial("trial-"+row.category,
    // ...
    newVar("trueFalseOption").global().set( v=>arraysTrialOptions[row.category].pop() )
    ,
    newScale("answer", "answer1", "answer2")
      .label(0, optionsCategories[row.category][0] ) // always use the same first answer
      .label(1, (optionsCategories[row.category][trueFalseOption==true ? 1 : 2]) )
      .center()
      .button()
      // ...
  )
  .log("trueFalseOption", getVar("trueFalseOption"))
)

Re. questions a/b, I think the codes in this post sort of answer them already but I invite you to read the documentation about how labels and Template work, and this tutorial page on using a table. As you can see, I cannot give one general answer to these questions, as it all depends on how specifically you want to address the other questions, but you can answer them once you know how generating trials from a table and referencing them using their label in Sequence works

Jeremy

[Jeremy]

Hi,

1. I’m not sure I understand: what’s wrong with the way you’re currently coding the design? (other than that you might want to remove the Canvas element instead of just the Image element, so that the Controller elements do not appear so far down on the page)

If you find it easier to code the two tasks that constitute a trial separately, you can always create two functions that each return an array of PennController commands, and call those functions inside the newTrial so the commands are executing sequentially, eg:

const pic = row => [
    newImage("pic", row.pic_name).size(720, 405).print()
    ,
    newCanvas("canvas_pic",720,405)
        .add( 0, 0, getImage("pic"))
        .center()
        .print()
    ,
    newTimer("displaytime", 2000).start()
    ,
    newKey("resp-flanker", "FJ")
        .log()
        .callback( getTimer("displaytime").stop() )
    ,
    getTimer("displaytime").wait()
    ,
    getCanvas("canvas_pic").remove()
];

const spr = row => [
    newController("DashedSentence", {s: row.sent_cont})
        .print()
        .log()
        .wait()
        .remove()
    ,
    newController("Question", {q: "Does it make sense?", as: ["Yes", "不合理"]})
        .print()
        .log()
        .wait()
        .remove()
];

Template("items.csv", row =>
    newTrial("experimental-trial",
        ...pic(row)
        ,
        ...spr(row)
    )
)

2. Do you mean that between each target trial (ie. pic+spr pair) you want to have 1-to-3 filler trials? And that you would have 20 occurrences of those filler breaks, and you want the total of filler trials to sum up to 40?

3. This will depend on the answer to question 2. If you have, say, 40 trials labeled “target” and 40 trials labeled “filler”, then rshuffle("target","filler") would give you just that: a series of 80 trials alternating between trials labeled “target” and trials labeled “filler”

On the other hand, if you have 20 target trials and 40 filler trials, and want to insert 1-to-3 filler trials after each target trial, which is how I understand your second point, then you could define this function to generate an array of random integers that sum up to a set value:

const nRandomIntsToSum = (length,minInt,maxInt,targetSum) => {
  const a = [];
  for (let i=0; i<length; i++) {
    let min = minInt, max = maxInt, remainder = targetSum-(a.reduce((x,y)=>x+y,0)+min*(length-(i+1)));
    if (remainder < maxInt) max = remainder;
    a.push( min+Math.round(Math.random()*(max-min)) );
  }
  fisherYates(a);
  return a;
}

Then you can use the custom pick function in 20 iterations, like this:

targets = randomize("target")
fillers = randomize("filler")
repetitions = nRandomIntsToSum(20,1,3,40)

Sequence(
  "instructions", 
  ...repetitions.map(n=>[ pick(targets,1),pick(fillers,n) ]).flat()
)

Jeremy

[Jeremy]

Hello,

I can see my answers in the results file when I test a copy of your experiment, here are the first four lines:

1665953582	REDACTED	PennController	0	0	teste	NULL	PennController	0	_Trial_	Start	1665953521474	NULL
1665953582	REDACTED	PennController	0	0	teste	NULL	Html	teste	case1	it	1665953582867	text input
1665953582	REDACTED	PennController	0	0	teste	NULL	Html	teste	case2	reux	1665953582867	text input
1665953582	REDACTED	PennController	0	0	teste	NULL	Html	teste	case3	nt	1665953582867	text input

Are you still unable to see the answers today?

Jeremy

[Jeremy]

Hi,

Use row to fetch content from your table, as in newTextInput("verb_answer", row.verb)

Jeremy

[Jeremy]

Hi,

Yes, this is described in the tutorials here and here

You can name the column “Group” or “List”, it doesn’t matter, PennController will use it to subset the table to the rows whose corresponding column contains only one of the possible values

Circulate the link to your experiment ending with server.py?withsquare=0 to some participants, with server.py?withsquare=1 to some other participants, and so on, until you get 10 submissions for each list

Jeremy

[Jeremy]

I’m not sure, if you refreshed the page it shouldn’t be a problem

FYI, I only see in the database 1 submission (from Oct 5) collected via the demonstration link (unpublished) for the project whose URL you shared earlier, and 10 submissions (4 from Oct 5, 5 from Oct 6, 1 from Oct 10) collected via the data-collection link (published)

Jeremy

[Jeremy]

Yes, you should always refresh the page before clicking the “Results” button. Also allow for some time for the results to be processed by the server (up to a day if the experiment logs lots of lines and/or the servers are being overloaded)

Jeremy

[Jeremy]

It works when I use ?id=Jeremy, eg https://farm.pcibex.net/r/LAfkkh/?id=Jeremy (I used a copy of your project and checked the results file) so the problem must be coming from the configuration on Prolific

As far as I can tell, the PCIbex document is up to date with Prolific’s documentation

Jeremy

[Jeremy]

Hi Weijie,

See this post for an explanation and possible ways to address the situation

Jeremy

[Jeremy]

Hi,

1. Starting with line 316 of Context_5y_mot_1.csv, some <br>s are replaced with [br]s

2. Each Template command browses the whole CSV file and outputs as many trials labeled accordingly. So your script outputs 120 trials, half labeled “trial” and half labeled “trial-Q”, which are duplicates of each other except for the addition of a comprehension question for the latter.

I think the part subsequence(repeat(randomize("trial"),4),"trial-Q") does something like, I randomly draw 5 trials from the 60 trials, and for these 5 trials, the first 4 trials are assigned to the “trial” template (with no comprehension question) and are randomized for order, and the last trial is assigned to the “trial-Q” template (with a comprehension question).

This is not how Template and trial references work. Each trial has a label, and you can refer to those labels in setting up the experiment’s sequence. Simply referencing "trial-Q" will refer to all the trials labeled “trial-Q” (60 trials in your case); randomize("trial") represents all the trials labeled “trial” (again, 60 trials in your case) but in a randomized order. repeat(randomize("trial"),4) represents a way of iterating over series of four trials from that set of 60 trials; inserting it inside subsequence will iterate until the series exhausts all the trials from the set. So subsequence(repeat(randomize("trial"),4),"trial-Q") will repeat a series of 4 (different) trials labeled “trial” followed by a (different) trial from the set of the 60 trials labeled “trial-Q”, until all the former have been drawn: in the end, you’ll have 15 subsequences of a series of 4 “trial” trials followed by one “trial-Q” trial, so all the 60 “trial” trials, and 15 “trial-Q” trials. Note that those 15 “trial-Q” trials will have been generated from the same CSV lines as 15 of the “trial” trials, so you’ll have duplicates, in that sense

If you know in advance you’ll have 60 trials, the easiest thing to do is create an array of 60 boolean entries, 15 true and 45 false, and shuffle that array. Then, you only generate 60 trials all labeled the same way, and pop the value at the top of the array for each trial: if it’s true then you include the question, otherwise you don’t.

arrayTrialQ = [...Array(60)].map((v,i)=>i<15)
fisherYates(arrayTrialQ)

Template("Context_5y_mot_1.csv" , row =>
  newTrial("trial",
    newVar("correct").global()
    ,
    newTimer("pre-trial", 500).start().wait()
    ,
    newText("target1", row.context).center().print()
    ,
    newText("line1", " ").center().print()
    ,
    newScale("answer", row.D1, row.D2) 
      .center()
      .button()
      .print()
      .log()
      .wait()
    ,
    getScale("answer")
      .test.selected(row.ans==row.D1 ? row.D1 : row.D2)
      .success( getVar("correct").set(true) )
      .failure( getVar("correct").set(false) )
      .remove()
    ,
    newTimer("post-trial", 500).start().wait()
    ,
    getText("target1").remove()
    ,
    getText("line1").remove()
    ,
    newVar("correctQ").global().set("NA")
    ,
    ...( arrayTrialQ.pop() == true ? [
      newText("target2", row.question)
        .center().print()
      ,
      newText("line2", " ")
        .center().print()
      ,
      newScale("answer-Q", "[Yes]", "[No]") 
        .center()
        .button()
        .print()
        .log()
        .wait()
      ,
      getScale("answer-Q")
        .test.selected(row.ansQ=='[Yes]' ? "[Yes]" : "[No]")
        .success( 
            getVar("correctQ").set(true),
        )
        .failure( 
            getVar("correctQ").set(false),
        )
      ,
      newTimer("post-Q", 500).start().wait()
    ] : [] )
  )
  .log("context", row.context)
  .log("Correct", getVar("correct"))
  .log("Question", getVar("correctQ"))
)

Then just use randomize("trial") in Sequence instead of the whole subsequence command. In the results file, the “Question” column will read “NA” for those trials without a comprehension question, and “true” or “false” for the ones with a comprehension question

Jeremy

[Jeremy]

Hi,

In order to really get an uninterrupted flow, you need to have simple text nodes surrounding the textarea node, and have both the textarea and its container’s display style set to inline. You won’t be able to achieve that by only PennController’s print (or before/after) command on Text elements, since it introduces embedded nodes

However, you can use the javascript functions after and before to insert simple text nodes around the textarea element:

newTextInput("filledInBlank")
    .size("6em", "1.5em")
    .lines(0)
    .css({display:"inline",outline:"none",resize:"none",border:0,padding:0,margin:0,"vertical-align":"-.33em","background-color":"yellow","border-bottom":"2px solid black"})
    .cssContainer("display","inline")
    .print()
,
newFunction( ()=>{
    const textinput = document.querySelector("textarea.PennController-filledInBlank");
    textinput.before("Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad  ");
    textinput.after(" veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.");
}).call()

Jeremy

[Jeremy]

Hi Ugurcan,

Do you still experience this issue? The experiment seems to be running smoothly on my end

I’m not sure why you created several new projects, but if it was in an attempt to work around the saving issue, you shouldn’t need to copy the whole project

Jeremy

[Jeremy]

Did you include ?id={{%PROLIFIC_PID%}} at the end of the URL, as described in step 3?

Jeremy

[Jeremy]

Hello,

You could technically force recordings to all be 3000ms, for example by chopping them if necessary, or maybe even by writing a code optimized well enough that it would practically always produce a 3000ms audio file on most if not all configurations, but I don’t think that is feasible within PennController as it is currently written: there are just too many computational steps going on around and in parallel to the recording process

Even if the recordings are elicited through the same template within one trial block, you have no guarantee that the cycles will always align perfectly. Say your browser notifies your Timer elements every 20ms. What if one Timer element for one trials starts 10ms after the start of a cycle, but the same Timer element for the next trial starts 15ms after the start of a cycle? You’ll have a 5ms difference. And that’s assuming that performance is constant throughout the experiment, which is an unreasonable assumption. Performance will vary depending on what tasks the browser (and the OS more generally) processes in parallel, and even the simple fact that as the experiment progresses, the browser needs to maintain audio recordings in cache can impact performance (although, admittedly, it shouldn’t have a noticeable impact under normal circumstances)

That being said, since the script of your experiment calls log on all the relevant elements in the trial, you can get a pretty good idea of when the various trial events happen wrt the audio recording. The Start timestamp of the MediaRecorder will very closely coincide with position 0 of your audio recording (maybe a few ms off, but it’s unlikely it would reach 10ms, unless the browser is lacking processing power). Then the Print timestamp of the “training_pic” Image element will inform you when the image becomes visible on the page, so subtracting the Start timestamp of the MediaRecorder from it will give you a very good approximation of how many milliseconds in the audio recording that event took place. Same thing for the “fixcross_training” Image element

At the end of the day, as it is currently written, PennController won’t provide you with the means to conduct analyses of events wrt audio recordings with a precision to the millisecond. For most psycho-linguistic questions, a precision to the centisecond is usually sufficient, but I have to concede that this is a limitation of PennController at the moment

Jeremy

[Jeremy]

Hi,

There is a typo in that code (that I apparently failed to fix): it should be newText("<p><a href='https://app.prolific.co/submissions/complete?cc=CGL9GN20'"+ GetURLParameter("id")+"' target='_blank'>Click here to confirm your participation on Prolific.</a></p> <p>This is a necessary step in order for you to receive participation credit!</p>") (ie. there should be a " after ', before + GetURLParameter)

Jeremy

[Author]

Posts