Group Columns – distribution

PennController for IBEX Forums Support Group Columns – distribution

This topic contains 16 replies, has 3 voices, and was last updated by Avatar Mogli 2 weeks, 1 day ago.

Viewing 15 posts - 1 through 15 (of 17 total)
  • Author
    Posts
  • #4120
    Avatar
    Mogli
    Participant

    Hello Jeremy,

    I have a question regarding the GROUP column in my csv file. I have 16 lists (Group A-P).

    If I have 12 participant, will Ibex distribute the lists, that in each case 2 test persons receive a list? How does the distribution work?

    Kind regards,

    Mogli

    #4122
    Jeremy
    Jeremy
    Keymaster

    Hi Mogli,

    Ibex cannot know in advance how many participants you will end up running, so it cannot automatically assign groups based on your total number of participants.

    Each participant is associated with one value from the Group column and only sees trials generated from rows with that value in the Group column. How that value is picked depends on how you set your experiment, as described in the original Ibex documentation. By default, PennController uses Ibex’s internal counter: in your case, your first participants would be assigned group A (the first value of the Group column in your spreadsheet) and after the first participant has completed your experiment, the next participant who takes the experiment would be assigned group B. After the second participant completes your experiment, the next participant taking the experiment would be assigned group C, etc.
    If you like the internal counter method but would rather have the counter incremented as soon as a participant starts the experiment, you can use PennController.SetCounter.

    You can replace experiment.html with server.py?withsquare=N in your experiment’s URL to force running the experiment in a specific group, where N is a number, from 0 to 15 in your case (see Ibex’s documentation).

    Let me know if you have any questions

    Jeremy

    • This reply was modified 3 weeks, 4 days ago by Jeremy Jeremy.
    #4130
    Avatar
    robpetrosino
    Participant

    I am going to jump in the conversation since I have an almost similar question.

    I have 6 word lists of the same rating experiment. Right now I have 6 different PCIbex projects with the same script, but different word lists (= 6 different csv’s); each experiment also has a number of fillers (they are the same across projects). As far as I could gather from your explanation, Jeremy, I can collapse the 6 lists in a single csv and add a “Group” column with the number from 1 to 6. I would then have to put PennController.SetCounter(); at the beginning of the experiment so that the counter increments as soon as a new participant start the experiment, but then PCIbex will deal with it. Is that correct? If so, I have three remaining questions:

    1. To store which group each subject has been associated with, I should just call the .log command for the corresponding column, right?
    2. In my code, the set of fillers is coded separately from the set of trials (i.e., they have different PennController.Template() calls) and they refer to different csv’s. Could this possibly mess with the grouping of the actual trials?
    3. Do you think that changing the URL and enforcing selection of a specific word list would be more reliable than exclusively relying on PCIbex? We are going to use AMT, so either way will work from the recruitment side; I was just wondering which method would be optimal.

    Thanks!

    #4132
    Jeremy
    Jeremy
    Keymaster

    Hi,

    It sounds like you can use a single CSV file within a single project indeed.

    1. Yes, see this page from the tutorial for a detailed discussion.

    2. All should be fine as long as you specify the right table for each Template command, and as long as your tables contain the same set of values in their Group columns (since PennController 1.4). That is, if trials are generated from rows where Group is A is table 1, trials will be generated from rows where Group is A in table 2.

    3. Yes, I think it would be more reliable and this is what I usually do: I collect one batch of participants in one group, then update the URL and collect the next batch of participants in another group.

    Let me know if you have more questions

    Jeremy

    #4134
    Avatar
    robpetrosino
    Participant

    Thanks for your message, Jeremy! I have a clarification question on this:

    2. All should be fine as long as you specify the right table for each Template command, and as long as your tables contain the same set of values in their Group columns (since PennController 1.4). That is, if trials are generated from rows where Group is A is table 1, trials will be generated from rows where Group is A in table 2.

    Does that mean that I will have to add a Group variable for the fillers csv’s as well? Since the fillers will have to be same for each subject regardless of the list each subject is assigned to, that would mean that I would have to replicate all the fillers for each of the list and assign each replication to a list. Is that correct? If so, I am tempted to stick to the original plan of having separate projects for each list and then use R to bind all results files together.

    #4135
    Jeremy
    Jeremy
    Keymaster

    Ah, sorry, I didn’t realize fillers were not grouped. Then no, you don’t have to duplicate the rows, just make sure you don’t have a Group column in you fillers table so Template doesn’t exclude any row.

    #4136
    Avatar
    robpetrosino
    Participant

    Oh okay, makes sense. Thanks!

    #4139
    Avatar
    Mogli
    Participant

    Hey,

    i have one last question before I finish my work. I had to separate my 16 lists into 1-8 and 9-16. Is it necessary to begin a Group column with the letter A or can it begin also with the letter I?

    Kind regards,

    Mogli

    #4140
    Jeremy
    Jeremy
    Keymaster

    It can start with anything, all that PCIbex cares about is the set of different values listed in the Group column: it will just pick one of them and automatically subset the table to the rows in which the Group cell contains the value it picked.

    Jeremy

    #4141
    Avatar
    Mogli
    Participant

    Hey Jeremy,

    the lists were not distributed 🙁 it picked all the time the same list. This is really a problem since the participants were paid. Do you have an idea why this happened?

    Mogli

    #4142
    Jeremy
    Jeremy
    Keymaster

    Hey Mogli,

    I’m sorry about that. What I do is I usually test-run my experiments so I can spot this kind of problems.

    There could be several explanations. The easiest for me would be if you shared your script or the link to your experiment with me.
    How was your group distribution method? Did you rely on the internal counter? Or did you force a group assignment as a parameter in the URL?
    When relying on the internal counter, and in absence of explicit manipulation in the script, if all your participants take the experiment within seconds/minutes, they could very well all end up in the same group (because the counter is only incremented when participants have completed the whole experiment).

    Jeremy

    #4143
    Avatar
    Mogli
    Participant

    Hey Jeremy,

    thank you for your help. I did a test run before.
    I rely on the internal counter. I used the csv file with the group column:

    List,Block,Item,expSet,grammatically_correct,condition,WP1_match,WP2_tense,SocStat_context,SocStat_contextVers,SocStat_text,Group,Sentence
    1,1,2,WP2,no,h,x,past,high,1,<p>Die folgenden Sätze wurden von einer Person geschrieben, die in München aufgewachsen ist und dort zur Schule gegangen ist.<br> Die Person arbeitet aktuell als Lehrperson an einem Gymnasium. Die Person spielt zwei mal die Woche Tennis.</br></p> ,A,Der Skinhead züchtigen den Sträfling.

    2,2,28,SocStat,yes,a,x,x,low,1,<p>Die folgenden Sätze wurden von einer Person geschrieben, die in Offenbach aufgewachsen und dort zur Schule gegangen ist.<br> Die Person arbeitet aktuell als Verkaufskraft in einem Obst- und Gemüseladen. Die Person geht zwei mal die Woche joggen.</br></p>,B,Ich mutmaße, dass der Park wegen der schmalen Größe übersichtlich ist.

    —–

    And this is my PennController Skript:

    PennController.Template( “ListeCSV.csv” ,
    variable => PennController(

    newText(variable.SocStat_text)
    .settings.bold()
    .settings.color(“blue”)
    .settings.center()
    .print()

    ,

    newText(variable.Sentence)
    .settings.center()
    .settings.bold()
    .print()

    ,

    newCanvas(“empty canvas53”, 1, 40)
    .print()

    ,

    newTimer(“wait”, 5000)
    .start()
    .wait()

    ,

    newText(“korrekt”, “Wie grammatikalisch korrekt finden Sie diesen Satz?”)
    .settings.center()
    .print()

    ,

    newScale(“korrekt”, 7)
    .settings.center()
    .settings.before( newText(“left”, “1 <br /> (trifft gar nicht zu)”) )
    .settings.after( newText(“right”, “7 <br /> (trifft voll und ganz zu)”) )
    .print()
    .settings.log()
    .wait()
    .remove()

    ,

    getText(“korrekt”)
    .remove()

    )
    .log( “List” , variable.List )
    .log( “Block” , variable.Block )

    )

    Again, thank you very much.

    #4144
    Avatar
    Mogli
    Participant

    I checked the times of the responses in my result file and I think you are right, that the participants ended up in the same group.

    #4145
    Jeremy
    Jeremy
    Keymaster

    Yes, it’s a general concern with Ibex’s default handling of the internal counter. Maybe next releases of PennController should replace the default behavior so it increments the counter at the very beginning of the experiment instead, but I’m afraid that people used to the original Ibex wouldn’t notice the change and make the wrong assumptions.

    In any case, you have multiple solutions:

    1. You can manually set the counter using PennController.SetCounter (place its label where you see fit in your sequence of trials, probably at the very beginning or immediately after the welcome screen)
    2. You can replace experiment.html in your experiment’s URL with server.py?withsquare=N where N is a number from 0 to the total-1 of different values appearing in your Group column. Then you have two options:
      1. You collect data in as many batches of participants as different N-based links (this is what I usually do: I collect one batch of participants, then update the link and collect another batch).
      2. You find a solution to automatically fill N, for example passing the participant’s ID in its place if the ID is a number: N is modulo-based so if your participants’ IDs are evenly distributed, you should get a reasonable output distribution.

      Let me know if you have questions

      Jeremy

    #4146
    Avatar
    Mogli
    Participant

    regarding your suggestion 2. a:
    This is my original link:
    https://korpling.german.hu-berlin.de/ibexexps/Mogli/clickmh/experiment.html

    and this is how you have suggested it:

    https://korpling.german.hu-berlin.de/ibexexps/Mogli/clickmh/server.py?withsquare=8.html

    Right? (8 for 8 participants and 8 lists, so every participant gets another list)

Viewing 15 posts - 1 through 15 (of 17 total)

You must be logged in to reply to this topic.