Comprehension question and rshuffle in self-paced reading

[JunLyu]

Dear Jeremy,

I’m moving my studies from Ibex Farm to PC Ibex. I’m currently designing an experiment and have some questions re: the presentation of comprehension questions (i.e., randomization of answer choices) and rshuffling the target items and fillers.

1. Answer choices

Here’s the code I have so far:

Template("demo_data.csv", row =>
    newTrial("experimental-trial",
    newController("DashedSentence",{s: row.Sentence})
        .print()
        .center()
        .log()      
        .wait()
        .remove()
    ,
    newScale(7)
        .before( newText("left", "<div class='fancy'>(highly unacceptable)</div>") )
        .after( newText("right", "<div class='fancy'>(highly acceptable)</div>") )
        .labelsPosition("top")
        .keys()
        .log()
        .once()
        .center()
        .print()
        .wait()
        .remove()
    ,
    newText("CompQuestion", row.Question)
        .center()
        .print()
    ,
   newScale("answer",row.Answer1,row.Answer2)
        .labelsPosition("right")
        .center()
        .log()
        .print()
        .wait()
        )
    )

For the newScale bit, I need randomization of presentation order for Answer1 and Answer2 from my template. Any suggestion?

2. rshuffling target items and fillers
In the old Ibex, here’s the code I use：

var shuffleSequence = seq("consent",..., sepWith("sep", rshuffle(startsWith("Target_"), startsWith("Filler_"))),"end");

In the PC Ibex, I guess I need something like:

Sequence("consent","instructions", "practice1","practice2","practice3","transfer",rshuffle("experimental-trial","filler"), SendResults(), "end")

But does that mean I need two templates/csv files, one for target items, another for filler items?

I’d really appreciate your help. Thank you!

[Jeremy]

Hi,

1. You could do newScale("answer", ...[row.Answer1,row.Answer2].sort(v=>0.5-Math.random())

2. You can reuse the exact same code in Sequence that you use in seq. If your question is about labeling your trials so they start by “Target_” or “Filler_” like they do when you manually list them in an array (following the native-Ibex syntax) the answer is simply pass the labels you want as the first argument of newTrial. If your two types of trials are listed in the same CSV table, it’s a good idea to include a column (named Type, for example) that reports the trial’s type, so you can do this:

Template("demo_data.csv", row =>
    newTrial( row.Type+"_trial" ,

Jeremy

[JunLyu]

Thank you Jeremy! The code for answer choice randomization works! However, I have one more question:

Do we need to specify in the csv file which answer is the correct one? We want to know if the participant has answered correctly or not and have the program assign a value to the participants’ responses (e.g., 1 as “hit”, 0 as “miss”). In the old Ibex, I only need to make the correct answer stay on the left position with the code as["Correct answer", "Wrong answer"], hasCorrect: true, randomOrder: true. How does that work in PC Ibex?

Thank you!
Jun

[Jeremy]

Hi Jun,

You could make Answer1 the correct answer in every row: since you shuffle the order in the scale anyway, your participants won’t be able to identify the correct answer by its order

Then if you don’t want to mark whether the participant’s answer was correct during post-data-collection analyses, you could use a global Var element to add a column that reports whether the choice was correct:

    newVar("isCorrect").global()
    ,
    newScale("answer",...[row.Answer1,row.Answer2].sort(v=>0.5-Math.random())
        .labelsPosition("right")
        .center()
        .log()
        .print()
        .wait()
        .test.selected( row.Answer1 )
        .success( getVar("isCorrect").set(true) )  
        .failure( getVar("isCorrect").set(false) )
  )
  .log("isCorrect", getVar("isCorrect"))
)

Jeremy

[JunLyu]

Thank you Jeremy! I tried the code below. I assume that although “global” in name, it still needs to be embedded within the “local” environment of newTrial? Is that correct? (I was thinking maybe a “global” variable should stand on its own, on a par with newTrial).

Template("demo.csv", row =>
    newTrial("experimental-trial",
    newController("DashedSentence",{s: row.Sentence})
        .cssContainer({"margin-top":"2em", "margin-bottom":"2em"})
        .print()
        .center()
        .log()      
        .wait()
        .remove()
    ,
    newText("CompQuestion", row.Question)
        .center()
        .print()
    ,
    newVar("isCorrect").global()
    ,
    newScale("answer",...[row.Answer1,row.Answer2].sort(v=>0.5-Math.random()))
        .labelsPosition("right")
        .center()
        .log()
        .print()
        .wait()
        .test.selected( row.Answer1 )
        .success( getVar("isCorrect").set(true) )  
        .failure( getVar("isCorrect").set(false) )
        .log("isCorrect", getVar("isCorrect"))
        )
    )

Jun

[Jeremy]

The command global makes the Var element accessible outside the trial where it is defined. Reference

Your code should work, but if you look back at the excerpt of code in my previous message, you’ll notice the .log command is indented more to the left than the other commands and is actually called on the closing parenthesis of newTrial and before the closing parenthesis of Template (although I omitted the Template and newTrial commands that come earlier in the code). Reference

Jeremy

[JunLyu]

Thank you Jeremy! That’s very helpful. I see your point: the .log("iscorrect",getVar("iscorrect")) bit is essentially appended to newTrial. I have two more very minor questions:

1. .success( getVar("isCorrect").set(true) ) sets the response to “true” when there is a correct response. Is “true” written as “1” to the results file, or is it written as “true” to the file? If it is the latter, can I set it to set("1")?

2. For one of my old Ibex files which I transferred to PC Ibex, I cannot turn the debugger off with DebugOff(). I wonder whether this is because the original Ibex syntax does not work well with DebugOff()? I have something like the following:

PennController.ResetPrefix(null); 

DebugOff();

Sequence("consent","instructions", "practice1","good_demo","practice2","bad_demo","practice3","more_demo","practice4","practice5","transfer",randomize("experimental-trial"), SendResults(), "end")

SetCounter("counter", "inc", 1);

newTrial(...

Jun

[Jeremy]

1. I think you will see true in the results file, but you should double-check. In any case, you can always true/false with 1/0 if you want

2. I made it impossible to turn the debugger off in the demonstration link, since people would sometimes turn it off while still working on their experiment and would miss warnings/errors, and end up spending much more time debugging their project. You can always click the X icon in the top-right corner of the debugger to not have it on the page, and the debugger will never show up using the data-collection link (DebugOff is no longer required)

Jeremy

[JunLyu]

Thank you Jeremy! I only checked the demonstration link, not the data collection link. It’s good to know!

Jun

[JunLyu]

Hi Jeremy,

Following my previous post, after data collection, I found that in the results file, the conditions (e.g., Target_a, Target_b,…) were not appended to the trials (I see “experiment-trial” appended to each trial instead), although the conditions were labeled in the csv file. The same problem applies to other labels like “Experiment”, “Group”, etc.

This the code I used:

Template("demo_data.csv", row =>
    newTrial("experimental-trial",
    newController("DashedSentence",{s: row.Sentence})
        .cssContainer({"margin-top":"2em", "margin-bottom":"2em"})
        .print()
        .center()
        .log()      
        .wait()
        .remove()
    ,
    newScale(7)
        .before( newText("left", "<div class='fancy'>(Highly unacceptable)</div>") )
        .after( newText("right", "<div class='fancy'>(Highly acceptable)</div>") )
        .labelsPosition({"margin-top":"2em", "margin-bottom":"2em"})
        .keys()
        .log()
        .once()
        .color("LightCoral")
        .center()
        .print()
        .wait()
        .remove()
        )
    )

Another minor issue is that for some reason, the Item ID for each trial was incremented by a fixed number (maybe by the number of practice trials and number of instruction texts).

Could you please let me know what went wrong? Thank you!

Jun

[Jeremy]

Hi Jun,

You need to use the newTrial().log command to append columns to your results lines. See an illustration in the advanced tutorial

Another minor issue is that for some reason, the Item ID for each trial was incremented by a fixed number (maybe by the number of practice trials and number of instruction texts).

Column #4 reports the order of creation of the trial: if you add newTrials above another newTrial then the latter will necessarily see its ID increase. Column #9 for each trial’s “Start” and “End” rows reports a similar information, except it applies to PennController trials exclusively

So, all things considered, nothing went wrong: you can control what information you add to each line by using newTrial().log, and you can decide to use or not use the information in column 4. Alternatively, if you have an “ID” column in the CSV table that you use to generate your items, you can append it to your results lines using .log and use that in your analyses instead of column 4

Jeremy

[JunLyu]

Thank you Jeremy for clearing my confusion! I can now get the information I want after a trial run. I have a minor clarification question regarding .log() though. I only need to append the .log() command after each NewTrial, correct? I don’t need to use .log() command after each “task” (spr, sentence-final judgment) I assume (although it should also work)?

For example, here’s what I used in the script (the .log() commands are appended to the end of newTrial):

Template("demo_data.csv", row =>
    newTrial("experimental-trial",
    newController("DashedSentence",{s: row.Sentence})
        .cssContainer({"margin-top":"2em", "margin-bottom":"2em"})
        .print()
        .center()
        .log()      
        .wait()
        .remove()
    ,
    newScale(7)
        .before( newText("left", "<div class='fancy'>(非常不通顺)</div>") )
        .after( newText("right", "<div class='fancy'>(非常通顺)</div>") )
        .labelsPosition("top")
        .keys()
        .log()
        .once()
        .color("LightCoral")
        .center()
        .print()
        .wait()
        .remove()
        )
    .log("group", row.Group)
    .log("item", row.Item_ID)
    .log("condition", row.Condition)
    )

Jun

[JunLyu]

Hi Jeremy,

Sorry for two more questions. I referenced the advanced tutorial (https://doc.pcibex.net/advanced-tutorial/11_collecting-participant-info.html) when embedding an html collecting the participants’ demographic information (age, gender, other languages, etc.). Here’s the bit of code I used:

newTrial("subject_info",
    newHtml("subject_info", "info.html") // info.html is a page collecting people's demographic info
        .cssContainer({"width":"720px"})
        .checkboxWarning("I agree to proceed")
        .print()
    ,
    newButton("continue", "click to continue")
        .center()
        .print()
        .wait(getHtml("subject_info").test.complete()
                  .failure(getHtml("csubject_info").warn())
        )
)

But I found that no participant’s information has been stored. Do I need to add another command to “log” their responses?

Another issue is that each participant has been assigned a label consisting of digits and letters. I thought these labels were unique for each participant (so that when I run my mixed model, each participant will be treated separately). But I think this is not always the case. Sometimes, two participants have been assigned the same label. I wonder when that might happen, and is there some good way to avoid this?

[Jeremy]

Hi again,

For example, here’s what I used in the script (the .log() commands are appended to the end of newTrial):

Yes, you are using newTrial().log correctly: it will add the values of row.Group, row.Item_ID and row.Condition to every single results line generated for the corresponding trial, whether for its start/end event, or for the Controller or Scale element

But I found that no participant’s information has been stored. Do I need to add another command to “log” their responses?

Yes, you need to use .log on the Html element to log its information

Another issue is that each participant has been assigned a label consisting of digits and letters. I thought these labels were unique for each participant

If you are referring to the values in the second column of the results file, they are “based on the subject’s IP address and various properties of their browser. Together with the value of the first column, this value should uniquely identify each subject.” If two submissions are sent using the same browser on the same device with the same IP, the MD5 hash will be the same for the two submissions. It is extremely unlikely, however, that the first column (Reception Time) would also have the same value, and if it does, either something went wrong and the same submission was submitted multiple times within a millisecond, or someone developed an algorithm to send multiple submissions in parallel; in either case, you should probably discard the submissions. Long story short: in the absence of a custom ID parameter, use the two first columns to uniquely identify your participants

Jeremy

[JunLyu]

Hi Jeremy,

It’s been a year since I last posted under this thread. I have tried randomizing the answer choices following what you suggested:

Template("file.csv", row =>
    newTrial("experimental-trial",
    newController("DashedSentence",{s: row.Sentence})
        .cssContainer({"margin-top":"2em", "margin-bottom":"2em"})
        .print()
        .center()
        .log()      
        .wait()
        .remove()
    ,
    newText("CompQuestion", row.Question)
        .center()
        .print()
    ,
    newVar("isCorrect").global()
    ,
    newScale("answer",...[row.Answer1,row.Answer2].sort(v=>0.5-Math.random()))
        .labelsPosition("right")
        .center()
        .log()
        .print()
        .wait()
        .test.selected( row.Answer1 )
        .success( getVar("isCorrect").set(true) )  
        .failure( getVar("isCorrect").set(false) )
    )
    .log("isCorrect", getVar("isCorrect"))
    .log("group", row.Group)
    .log("item", row.Item)
    .log("itemID", row.Item_ID)
    .log("condition", row.Condition)
)

But the presentation order of the answer choices is still fixed. Answer1 always appears before Answer2. Is there something I am missing?

Best,
Jun

[Author]

Posts