[Jeremy]

Hi again,

I simply added the code thing because the script your assistants sent me had a placeholder for it but no actual algorithm to deal with it. You should of course feel free to use your own method, especially if it was specifically designed for MTurk.

I have not used MTurk in a very long time now and have been using Prolific instead to recruit participants, which offers extensive screening options for researchers and does not charge a fee if you need to recruit more participants. What I do to keep track of participants is simply include id={{%PROLIFIC_PID%}} in my experiment’s URL on Prolific, and Prolific will automatically replace {{%PROLIFIC_PID%}} with a unique ID for that participant. Then in my experiment, I use .log as I mentioned in my previous message, in which I use PennController.GetURLParameter("id") to fetch the value of id from the URL. So in my projects, I usually do not generate a unique ID using javascript.
Last thing I do is include a confirmation link to Prolific on the last screen of my experiment.

When I was still using MTurk, I used the method I illustrated here: each participant is assigned a unique ID by the two lines from my message above, which gets recorded in their results file (again, using .log) and participants are invited to copy-paste the code on MTurk so I can then match the codes in my results files and those sent to MTurk by the participants and validate their submissions (or reject them if the code they entered on MTurk does not appear in my results file).

The tutorial describes yet another method, which can also be used with MTurk, where participants are invited to type a unique ID on the first page of the experiment (e.g. their MTurk ID).

Jeremy

[Jeremy]

Hi Leo,

Glad to hear you’ve solved the problem.

Re. the code variable: the two lines from my previous message are the ones generating the code, and they appear outside a PennController trial in the script I sent your assistants, which means they are immediately executed when your participants open the experiment. The call to a newFunction element in the “consent” chunk (which should be the very first screen your participants see) is just to make sure that the value of code is inaccessible from the javascript console upon runtime, so participants cannot simply type code in it to see their MTurk code and validate their submission early.
As for how the value of code appears in your results file, it’s just standard uses of the .log command on the PennController trials for which to record it (all of them, in the script I sent).

Let me know if you have any questions

Jeremy

[Jeremy]

Hi Leo,

If you are referring to a string of ~30 alphanumeric characters, it’s a confirmation code for validation on MTurk that is generated (and commented on) in the script I gave your assistants. I report the two lines generating it here:

code = a => a?(a^Math.random()*16>>a/4).toString(16):([1e7]+-1e3+-4e3+-8e3+-1e11).replace(/[018]/g,code)
code = code()

It is not a new PennController thing, I copied (and slightly adapted) it from https://stackoverflow.com/a/7061193

Jeremy

[Jeremy]

Hi Cassandra,

The labeling and sequence order system did not change in PennController, the only difference is that you no longer have to write var shuffleSequence = seq( ... ), you can use PennController.Sequence( ... ) instead. Items are labeled using the first argument of the PennController function that creates them (if provided as a string). Here’s an example (I use AddTable for the sake of illustration—using a CSV file works just as well):

PennController.AddTable( "myTable" , `Column1,Column2
UpperCase,Hello
LowerCase,hello
UpperCase,World
LowerCase,world`
)

PennController.Sequence( rshuffle("UpperCase","LowerCase") )

PennController.Template( "myTable" ,
  row => PennController( row.Column1 , // Using Column1 to label the trials
      newButton( row.Column2 )
        .print()
        .wait()
  )
)

Let me know if the example is unclear

Jeremy

[Jeremy]

Hi David,

Yes it’s definitely possible. Simply use two different CSV tables, and pass their name as the first parameter to your Template function. Example:

PennController.AddTable( "table1" , `Column1,Column2
Table1RowACol1,Table1RowACol2
Table1RowBCol1,Table1RowBCol2`)

PennController.AddTable( "table2" , `Column1,Column2
Table2RowACol1,Table2RowACol2
Table2RowBCol1,Table2RowBCol2`)

PennController.Sequence( randomize("forced") , randomize("likert") ) 

PennController.Template( "table1" ,
  row => PennController( "forced" ,
    newScale( "answer" , row.Column1 , row.Column2 )
        .settings.button()
        .settings.center()
        .print()
        .wait()
  )
)

PennController.Template( "table2" ,
  row => PennController( "likert" ,
    newScale( "score" , 5 )
        .settings.center()
        .settings.before( newText(row.Column1) ) 
        .settings.after( newText(row.Column2) )
        .print()
        .wait()
  )
)

This example uses AddTable but of course you can directly upload CSV files to your project and use their filenames

Let me know if you have questions

Jeremy

[Jeremy]

Oh, I see… Well, I’m afraid there is no straightforward adaptive solution. What you can do though is tweak the CSS of the labels to set a fixed width for the left and right labels, so they always both occupy the same space around the scale:

newText("Hello my dear dear friend, how are you doing on this very pleasant day?")
    .print()
,
newImage("bad", "no.png")
,
newImage("good", "ya.png")
,
newText("left label", "Baaaaaaaaaad")
    .settings.before( getImage("bad") )
    .settings.cssContainer({width: "30vw", "text-align": "right"})
,
newText("right label", "Good")
    .settings.after( getImage("good") )
    .settings.cssContainer({width: "30vw", "text-align": "left"})
,
newScale("judgment", 2)
    .settings.before( getText("left label") )
    .settings.after( getText("right label") )
    .settings.center()
    .print()
    .wait()

Of course you should feel free to play with the specific width you use (whether in vw/vh, px, em, etc.)

Jeremy

[Jeremy]

Hi Cassandra,

What is the problem exactly? The radio buttons always appear centered on my screen when I test the script on the document page you gave the link to, even if I use a 2-point scale instead of a 5-point scale and use "aBada" and "aGooda" instead of the "Bad" and "Good" as labels.
Admittedly, you might need to use settings.center depending on your aesthetics configuration. For example, using the test page from the documentation, the scale will left-align to the left-edge of the content box of the page, but settings.center takes care of it. Here’s a script if you want to try it (uncomment the settings.center line):

newText("Hello my dear dear friend, how are you doing on this very pleasant day?")
    .print()
,
newImage("bad", "no.png")
,
newImage("good", "ya.png")
,
newText("left label", "Bad")
    .settings.before( getImage("bad") )
,
newText("right label", "Good")
    .settings.after( getImage("good") )
,
newScale("judgment", 2)
    .settings.before( getText("left label") )
    .settings.after( getText("right label") )
    // .settings.center()
    .print()
    .wait()

Jeremy

[Jeremy]

Hi Leo,

I see, it’s actually impressive that participants can perform above average using a translator! Good to know, then I guess it would be worth adding a note asking participants not to use such tools when taking the experiment (some participants might be well-intentioned and not realize that it could be detrimental to data quality).

There is no straightforward implementation of the HTML canvas method, as canvas require you to code-draw their content in javascript. So you would need to add a canvas element onto the page somehow and then fill it using some javascript code. You can pass HTML to basically any string of a PennController element that ends up being printed, for example Text elements. And you can use newFunction to create (and then call) a javascript function. So here is a basic example (assuming you are inside a Template function, using row.Sentence to refer to the Sentence cell):

newText("canvasText", '')
  .print()
,
newFunction(()=>{
  let width = $(".PennController-canvasText-container").width();
  let cvs = document.getElementById("myCanvas");
  let ctx = cvs.getContext("2d");
  ctx.font = '36px serif';
  let words = row.Sentence.split(' '), lines = [], line = [];
  while (words.length){
    line.push(words.shift());
    if (ctx.measureText(line.join(' ')).width >= width){
      words = [line.pop(), ...words];
      lines.push(line);
      line = [];
    }
  }
  if (line.length)
    lines.push(line);
  cvs.width = width;
  cvs.height = 40 * lines.length;
  ctx.font = '36px serif';  // Needed for re-calculating px after resizing cvs
  for (let n = 0; n < lines.length; n++)
    ctx.fillText(lines[n].join(' '), 1, 38*(n+1));
}).call()

As you can see it's clearly not as elegant and concise as newText("blablabla").print(): because you're printing the text as an image, you have to manually calculate sizes and print each line, something browsers automatically do for you when the text is printed normally onto the page. But this code is still very basic: it will calculate sizes and positions based on what the page looks like at the exact moment when the Function is called, so it won't automatically adjust to any resizing after that. But at least it should be functional and definitely prevent automatic translation from extensions (again, the text is just an image---the extension would need a neural network or something to read the image).

Let me know if you have questions

Jeremy

[Jeremy]

Hi Leo!

I’m curious, how did you discover that your participant used an automatic translation extension?

I think all the solutions you list are quite ingenious. If you don’t mind showing text as images as your mention of screenshots seems to suggest (note that it would mean constant printing size on the screen, irrespective of the participant’s window size/resolution) then using the HTML canvas could be a good idea (not to be confused with PennController’s Canvas element). You can print text onto it using fillText and extensions should be blind to it.

I guess you could also use the intrusive character method, but how to best implement it would depend on how the extension works.

I’m not sure there would be a unique, ideal solution to implement as a feature of PennController for future releases. But really, I don’t how prevalent this problem is. I don’t recall noticing participants using an automatic translation tool (though I could have been fooled over and over again). Hopefully any such participant eventually gets filtered out by my exclusion criteria when it comes to data analysis. I guess getting a sense of how serious a problem it is would help you see whether it’s worth spending the time and effort coming up with and implementing a preventive solution.

Let me know if you need help with the canvas method and I can try to work something out for you.

Jeremy

[Jeremy]

Hi Cassandra,

Glad to hear you like PennController!

You can set the position of the text labels of the radio buttons using the command settings.labelsPosition. You can also use settings.before and settings.after to insert Text elements to the left or to the right of the scale, as in this example.

If you need to add some spacing to the left of the scale, you can try playing with CSS as explained in the Aesthetics documentation page. Or you could use try using the settings.css command, like this:

newScale(10)
    .settings.css("margin-left","1em")
    .settings.before(newText("Score"))
    .print()

I will print a 10-button radio scale with the text Score to its left, with 1em (~1 character width) of horizontal spacing between the left edge of the scale and the right edge of the text.

Let me know if you have any questions

Jeremy

[Jeremy]

Hi Itamar,

When defining a sequence of trials manually (in your case, using the Sequence command) you need to make sure that all your trials are properly labeled for them to be executed in the order you stipulate. The command SetCounter actually creates a trial, which is no exception to this rule.

I think the following should accomplish what you want:

PennController.ResetPrefix(null);
PennController.DebugOff();
PennController.SetCounter("increase");

PennController.Sequence( "increase", "welcome", "practice1", "practice2", "practice3", randomize("experiment"), "goodbye" );

If you don’t manually define the sequence of trials (i.e. no Sequence command and no definition of the internal Ibex shuffleSequence variable) then the trials are run in the top-down order in which they appear in your script. In your case, since SetCounter appears at the top of your script, it would be the first trial to be executed (if you had no Sequence command—are you sure this isn’t a recent addition?).

Let me know if that solved your issue and if you have any questions

Jeremy

• This reply was modified 5 years, 7 months ago by Jeremy. Reason: Added a note about absence of Sequence

[Jeremy]

Hi Sarah,

You’re almost there, but unfortunately you can’t refer to other PennController elements in a new* statement. And since you probably want to log the random code as well, I would store it in a Var element (whose .set command can also run javascript code):

let alphanum = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXTZabcdefghiklmnopqrstuvwxyz"; 

PennController(
    newVar("ID", "")
        .settings.global()
        .set(v =>[...Array(8)].reduce(a=>a+alphanum.charAt(Math.floor(Math.random()*alphanum.length)),''))
    ,
    newText("code", "")
        .settings.text( getVar("ID") ) 
        .print()
)
.log("code", getVar("ID"))

I made the ID var global so you can add its value at the end of your results lines using .log.

NB: I reused your code but of course the trial defined here will be fully executed (and done) in an instant unless you add a wait command on an element of an appropriate type.

Jeremy

[Jeremy]

Hi Sahar,

You probably want to add .log("Word", variable.Word) to your stack of logs. Calling .settings.log on the Text element itself will just record the timestamp at which it was printed on the screen. It’s true that the line in the results file should also show the name of the Text element, but note that newText(variable.Word) creates a text element whose content is the value of the Word cell, its name is basically a random string. If you want the same name and content, you can do newText(variable.Word,variable.Word).

Also, you might be interested in the Scale element to provide your participants with a 1-to-5 type of input.

Let me know if you have any questions

Jeremy

[Jeremy]

Hi Adam,

Just to let you know, I’ve released PennController 1.6, which fixes the logging issue of the Key element, so the code above would properly report the timestamps of each press on the space key (see the the documentation for installation/update instructions).

Jeremy

[Jeremy]

Hi Adam,

The latter method is the correct one, you just need to properly label your items so you can refer to them in Sequence:

PennController.ResetPrefix(null);

PennController.Sequence( randomize("dashed") , "send" );

PennController.Template( PennController.GetTable("mystimuli.csv") ,
    row => [
        "dashed", "DashedSentence", {s: row.Sentence},
    ]
);

PennController.SendResults("send");

This will only work starting with PennController 1.4 though (newly created experiments on expt.pcibex.net currently come with version 1.5).

NB: you don’t need to use GetTable if you’re not filtering your table, you can directly pass the string "mystimuli.csv" as the first argument of Template.

Jeremy

[Author]

Posts