Keeping track of participant accuracy

[miriam]

Hello!

I’m planning a self-paced reading study and want to keep track of participants’ response accuracy on comprehension questions presented after certain trials. I’m using Template to read in the trials from a .csv file.

I have tried to keep track of accuracy by defining a global variable (newVar(“PracAcc”).settings.global()) and then updating it on every trial, but if I initialize this variable in Header, it gets reset in every trial, whereas if I initialize it outside of Template or Header, the variable cannot be accessed. How can I keep track of participant accuracy throughout the experiment?

Many thanks in advance for your help!
Miriam

[Jeremy]

Hello Miriam,

Both options should normally work. Here is an example script, switch the commenting // to test out the two options:

PennController.ResetPrefix(null)

AddTable("myTable", "Correct\nYes\nNo\nYes\nNo" )

Header( 
    // newVar("Acc", []).global() 
)

newTrial( "intro" ,
    newVar("Acc", []).global(),
    newButton("Start").print().wait()
)

Template( "myTable" , row => newTrial( "exp" ,
    newVar("computedAcc").set(getVar("Acc")).set(v=>v.filter(a=>a===true).length/v.length),
    newText("accuracy").text(getVar("computedAcc")).print("left at 5px", "top at 5px")
    ,
    newScale("answer", "Yes","No").button().print().wait()
        .test.selected(row.Correct)
        .success( getVar("Acc").set(v=>[...v,true]) )
        .failure( getVar("Acc").set(v=>[...v,false]) )
))

newTrial( "end" ,
    newVar("computedAcc").set(getVar("Acc")).set(v=>v.filter(a=>a===true).length/v.length),
    newText("accuracy").text(getVar("computedAcc"))
    ,
    newText("Final accuracy: ").after(getText("accuracy")).print()
    ,
    newButton().wait()
)

In this example I set the global Var element to an array of true‘s and false‘s so I can then compute the current accuracy on each trial by dividing the number of true‘s by the length of the array.

Note that some Var-related bugs were fixed with PennController 1.8, so make sure to update your version of PennController if the script above does not work for you

Jeremy

[carlap]

I have a similar issue. I want to keep track of correct answers and I tried to integrate the information above but without success. I am not using a scale, I am using the arrow keys. My code to track the keys is:

newKey(“pressOnArrow”, “ArrowLeft”, “ArrowRight”) .callback( getTimer(“timeout”).stop() ).log(“all”) , newTimer(“timeout”, 5000).start().log().wait() , getKey(“pressOnArrow”) .disable() .test.pressed().success( newText(“Good job”).print() ) .failure( newText(“Too slow”).print() ))

Obviously that is not my original coding (thanks for the earlier help). Now, I want to track if the ArrowLeft or ArrowRight is the correct answer and store it in a variable to display at the end of a section of trials. I tried integrating:
.test.selected(row.Correct)
.success( getVar(“Acc”).set(v=>v+1) )

But it does not seem to work. I do not need to keep track of wrong answwers. Any thoughts? Thanks!

[Jeremy]

Hi,

There is no selected test for Key elements, what you are looking for is test.pressed

Example:

newTrial(
    newKey("test", "ArrowLeft", "ArrowRight")
        .wait()
        .test.pressed( "ArrowLeft" )
        .success( newText("Left").print() )
        .failure( newText("Right").print() )
    ,
    newButton().wait()
)

Jeremy

[miriam]

Thanks a lot Jeremy, both solutions work perfectly for me!

[miriam]

Hi Jeremy,

I now ran into a new problem when trying to access my accuracy variable during the experiment to provide feedback. Basically I just want to check whether the value lies between certain thresholds, but the statement below always seems to evaluate to false and print “Low accuracy”, even if I manually set accuracy to a higher value. Do you know how to fix this?

newTrial("feedback",
    newVar("Acc", 0.8).global()
    ,
    newText("low_acc", "Low accuracy")
    ,
    newText("high_acc", "High accuracy")
    ,
    (
      getVar("Acc") > 0.5 ? [
        getText("high_acc").print()
      ] : [
        getText("low_acc").print()
      ]
    )
    ,
    newButton("Exit")
        .center()
        .print()
        .wait()
)

[Jeremy]

Hi,

You are using a plain javascript ternary conditionals where you should be using a PennController test command. I briefly explain this point below the code in this message.

Your trial should look like this:

newTrial("feedback",
    newVar("Acc", 0.8).global()
    ,
    newText("low_acc", "Low accuracy")
    ,
    newText("high_acc", "High accuracy")
    ,
    getVar("Acc").test.is( v => v > 0.5 )
        .success( getText("high_acc").print() )
        .failure( getText("low_acc").print() )
    ,
    newButton("Exit")
        .center()
        .print()
        .wait()
)

Jeremy

[miriam]

Awesome, thank you! I had also tried using test.is() before but somehow it did not produce the intended results then. Now everything works 🙂

[k.stone]

Hi Jeremy & co,

I’m probably doing something very silly – but I’ve been using the code under miriam’s comment to store accurate responses. But I can’t seem to create the global variable correctly. I’ve tried:

newTrial( “intro” ,
newVar(“AccSubj”, 0).global(), // I don’t need a list, just a counter
newText(“debugInit”, “AccSubj = ” + getVar(“AccSubj”).value).print(),
newTimer(1000).start().wait()

)

–> result of debugInit is 0 as expected, but as soon as I start a newTrial (e.g., for an experiment), AccSubj is undefined. For example:

newTrial( “debug1” ,
newText(“debug11”, “AccSubj = ” + getVar(“AccSubj”).value).print(),
newTimer(1000).start().wait()

)

–> result “undefined”

Any tips?

Best,
Kate

[Jeremy]

Hi Kate,

You cannot reference the Var element in newText, since newX commands are not evaluated upon runtime.

What you can do is use the Var.set and Text.text commands like this:

newTrial(
    newVar("AccSubj", 0).global(),
    newVar("AccSubjText").set( getVar("AccSubj") ).set( v => "AccSubj = "+v ),
    newText("debugInit", "").text(getVar("AccSubjText")).print(),
    newButton("Increase").print().wait(),
    getVar("AccSubj").set(1)
)

newTrial(
    newVar("AccSubjText").set( getVar("AccSubj") ).set( v => "AccSubj = "+v ),
    newText("debugInit", "").text(getVar("AccSubjText")).print(),
    newButton("Increase again").print().wait(),
    getVar("AccSubj").set( v => v + 1 )
)

newTrial(
    newVar("AccSubjText").set( getVar("AccSubj") ).set( v => "AccSubj = "+v ),
    newText("debugInit", "").text(getVar("AccSubjText")).print(),
    newButton("").wait()
)

Jeremy

[Author]

Posts