PennController for IBEX › Forums › Support › Lists randomization
Tagged: list
- This topic has 9 replies, 5 voices, and was last updated 2 years, 4 months ago by
Florian Schwarz.
-
AuthorPosts
-
October 14, 2022 at 7:14 am #9568
rsw123
ParticipantHi Jeremy,
My question is about how to randomize the lists in my script.
The experiment has four lists: List1, List2, List3 and List4. And to say, I want to assign each list of 10 participants (40 in total). I was wondering how could I achieve this?
Should I include one column in the csv file called List? and how could I write the Sequence line?
Many thanks in advance!
October 14, 2022 at 4:04 pm #9570Jeremy
KeymasterHi,
Yes, this is described in the tutorials here and here
You can name the column “Group” or “List”, it doesn’t matter, PennController will use it to subset the table to the rows whose corresponding column contains only one of the possible values
Circulate the link to your experiment ending with
server.py?withsquare=0
to some participants, withserver.py?withsquare=1
to some other participants, and so on, until you get 10 submissions for each listJeremy
October 18, 2022 at 6:39 am #9585rsw123
ParticipantMany thanks for the link! I got the idea.
But before continue to the randomization step, I tried to run the following script and it returned the error message “Alert from preview: ERROR: There must be some items in the running order!” Things went well before adding the “practice” part. Could you please give me some hints on what happening here? (the demo link also attached here: https://farm.pcibex.net/r/QzMAyX/)
// Remove command prefix PennController.ResetPrefix(null) // Control trial sequence Sequence("information", "consent", "consentform", "demographic", "instructions", "practice", randomize("experimental-trial"), "send", "completion_screen") // Information sheet newTrial("information", newHtml("informationsheet", "informationsheet.html") .cssContainer({"width":"720px"}) .print() , newButton("continue", "Click here to continue") .center() .print() .wait(getHtml("informationsheet").test.complete() ) ) // Consent info newTrial("consent", newHtml("consent", "consent.html") .cssContainer({"width":"720px"}) .checkboxWarning("You must consent before continuing.") .print() , newButton("continue", "Click here to continue") .center() .print() .wait(getHtml("consent").test.complete() .failure(getHtml("consent").warn()) ) ) // Consent form newTrial("consentform", newHtml("consentform", "consentform.html") .cssContainer({"width":"720px"}) .checkboxWarning("You must consent before continuing.") .print() , newButton("I consent to items above") .center() .print() .wait(getHtml("consentform").test.complete() .failure(getHtml("consentform").warn()) ) ) // demographic newTrial("demographic", newHtml("demographic", "demographic.html") .cssContainer({"width":"720px"}) .checkboxWarning("You must consent before continuing.") .print() , newButton("continue", "Click here to continue") .center() .print() .wait(getHtml("demographic").test.complete() .failure(getHtml("demographic").warn()) ) ) // instructions newTrial("instructions", newHtml("instructions", "instructions.html") .cssContainer({"width":"720px"}) .checkboxWarning("You must consent before continuing.") .print() , newButton("continue", "Click here to continue") .center() .print() .wait(getHtml("instructions").test.complete() .failure(getHtml("instructions").warn()) ) ) // practice newTrial("practice", // will print all Text elements, horizontally centered defaultText.center().print() , newText("Practice Session") , newText("You will now be given eight practice trials to familiarize yourself with the experiment. During the practice trials, you will receive feedbacks on whether you recreate the pattern correctly and on whether your decision about the object is correct.") , newText("Please press 1 if the object has been mentioned in the preceding sentence, or press 2 if has not been mentioned.") , newText("You should do this as quickly and accurately as possible.") , newText("When you are ready, press C to start a practice run.") , newKey("keypress", "C") .wait() // Finish trial upon press on c )
October 19, 2022 at 6:36 pm #9593Jeremy
KeymasterHi,
The script that’s actually running has
PennController.ResetPrefix(null)
too far down (it should be the first line of the script) and it misses a comma between"demographic"
and"instructions"
in theSequence
commandThis is most likely due to the Save function bug on the farm: try copying the content of your script file, deleting it from your project and recreating it and pasting the content back in, and it should work
Jeremy
November 8, 2022 at 10:53 am #9694ediachek
ParticipantHi Jeremy,
Just launched my experiment yesterday for some pilot data and it works great! Thank you for working on the bugs and for your continued support.
I noticed just one thing — I have 2 phases of the experiment: study and test. And I also have 16 experimental lists, and participants are assigned to a list randomly through the SetCounter(“counter”, “inc”, 1); command at the beginning of the experiment. I assumed that participants will be assigned to the same study and test lists but I’m seeing that some participants completed list 3 for study and list 11 for test.
Is there a way to fix that so that each person completes the same list for study and test?
Here is the link to the expt: https://farm.pcibex.net/r/dmrkYm/
Best,
Yev-
This reply was modified 2 years, 5 months ago by
ediachek.
November 8, 2022 at 3:01 pm #9699Jeremy
KeymasterHi Yev,
Your two tables to have 16 groups each, but only 7 groups are shared by the two tables (
10
,11
,12
,13
,14
,15
,16
). The other 9 groups are different:1
(study) vs1
(test),2
(study) vs2
(test), etc.Because
group
for example in the study table is2
, but10
in the test table (because1
comes before2
and0
comes before1
). So any participant running with a counter value of 2 (or modulo 2) will be assigned group2
from the study table, and group10
from the test tableJeremy
November 8, 2022 at 3:39 pm #9702ediachek
ParticipantHi Jeremy,
Thank you! I would never be able to figure this out.
So I tried deleting the (invisible?) space character using Excel functions but I’m not confident it worked, so I’ve decided to assign numbers 51-59 to lists 1-9. Would that work do you think?
Best,
YevNovember 8, 2022 at 4:24 pm #9703Jeremy
KeymasterAll you need is to make sure you have the exact same possible values across the two tables. If you replace all the 1-digit group values from both tables with numbers starting with
5
then you could achieve that, as long as you don’t introduce another space character in front of5
in one of the two tablesThe other solution would be to open the study table in a text editor (for example, the editor on the farm) and delete the space character in front of all the 1-digit group values
Jeremy
December 16, 2022 at 9:05 am #9798ajessen
ParticipantHi Jeremy,
I have a question regarding the distribution of the lists as well.
I ran an experiment which had 12 lists, and lasted about 40 minutes. It was published via prolific. Since there participants often fill up quickly, I set the counter earlier than at the end of the experiments, so that not everybody signing up within the first participant doing the experiment will get list1.
The counter was set after the training session and after they got ask to click a button for the experiment to start. I figure that there is a big chance that participants who got that far will also finish it – but i am aware that not everybody might do so.
When checking the results, it showed that the distribution of the lists was kind of erratic. It started with 3, then 5, 7, 9, then again 3, 5, maybe 6, 10, then again 3, without going at least until 11 or 12. 1 and 2 left out.
We changed the uploaded excel file to delete all lists that where presented already more than 2 (we wanted just 2 participant per list), but I hope there is another way to make sure, that the distribution of the lists is at least a bit more reliable.Here is the code:
Sequence( "welcome" , "consent", "instruction", "practice1", "practice2","send", "trialTriangle", "start experiment", "counter", "experimental-trial", "send", "final"); [all elements in between] newTrial("practice2", newText("top", "zgomot") , newText("left", "copil") , newText("right", "speria") , // newImage("triangle", "Triangle.png") // .size(140,140) //, newCanvas("trialTriangle", 200,200) .settings.add( 50,0, getText("top").color("blue")) .settings.add(-30,140, getText("left")) .settings.add(120,140, getText("right")) .center() .print() , newTextInput("response", "") .center() .log() .print() .wait(getTextInput("response").testNot.text("")) ); newTrial("start experiment", defaultText.center().print(), newText("Când sunteți gata să începeți experimentul, apăsați SPAȚIU.") , newKey(" ").wait() // Finish trial upon press on spacebar ); SetCounter("counter", "inc", 1); Template("draft_list_2.csv", row => newTrial("experimental-trial", newText("top", row.TOP).color(row.Coloring_Code=="TOO" ? "blue":"black"), newText("left", row.LEFT).color(row.Coloring_Code=="OLO" ? "blue":"black"), newText("right", row.RIGHT).color(row.Coloring_Code=="OOR" ? "blue":"black"), newCanvas("trialCanvas", 200,200) .settings.add( 50,0, getText("top")) .settings.add(-30,140, getText("left")) .settings.add(120,140, getText("right")) .center() .print() .log(), newTextInput("response", "") .center() .log() .print() .wait(getTextInput("response").testNot.text("")) )
and here the link to the full experiment
https://farm.pcibex.net/r/DYuHqf/
it’s the demonstration linkThank you very much!!
Best, Anna
December 20, 2022 at 7:17 pm #9800Florian Schwarz
ModeratorHi Anna,
(Jeremy is traveling right now and may not be able to answer before a while, so I’m jumping in.)There unfortunately is no fool-proof way of ensuring equal number of submissions across lists in an automated way. The only method giving you full control is to post separate experiments on Prolific (with restrictions set to allow people not to take more than one) with urls to your experiment that hard-code the group/list-assignment, in the way Jeremy described above:
“Circulate the link to your experiment ending with server.py?withsquare=0 to some participants, with server.py?withsquare=1 to some other participants, and so on, until you get 10 submissions for each list”
Then you can make sure that each list gets to the desired number of participants.
Hope that helps,
Florian -
This reply was modified 2 years, 5 months ago by
-
AuthorPosts
- You must be logged in to reply to this topic.