[leaena]

Hi Jeremy,

Never mind, apparently the problem occurred between accessing the data and saving it, or else between saving it and importing it into R.

Thanks for your help!
Leo

[leaena]

Hi Jeremy,

I thought I should update you on this in case you are interested. We eventually came up with a python script that inserts html in between each letter and either changes the color to slightly-off-black-but-undetectably-so, or inserts formatting that doesn’t show up (e.g. <span>). This breaks the auto-translation plugin. It doesn’t prevent copying and pasting or image recognition, but we’ve deemed it good enough for our purposes for the time being.

Best,
Leo

[leaena]

Hi Jeremy,

How are we intended to be using the validation code? As it is, we can use it as a unique participant identifier (if I’m understanding it correctly) which we were doing before by the unique combinations of md5Hash and timeReceived. However I am curious to know what validation method you are using and how, since the posting method I am using right now is this: https://github.com/jhupsycholing/mturk-microbatcher which is an app written by Brian Leonard at U Rochester, that posts 9 at a time (for fee reasons) and does other things like exclude blacklists and duplicate IP addresses, but which unfortunately does not allow for individual participant code validation. I am not sure how I would use a unique identifier like that even if we were not using this batcher, though – we used to use randomly generated numbers. So I am interested to know what use this code is intended to be put to and how that works.

Thanks for anything I can learn! Much appreciated.
Leo

[leaena]

Actually – we may have fixed this. Apparently they had to come after the “set number correct” thing but they are displaying now. Thank you!

Leo

[leaena]

Hi Jeremy,

Thank you for getting back to me. Unfortunately we’re running into a problem. The “code” code seems to be working, as in it sends a value to the results file. However you call it in the “consent” chunk, which you redefined for them. However, when we try to include the “consent” or “metadata” chunks (two html forms which as far as we can tell are coded to run exactly the same way as the instructions, except they are forms that require input from participants), the experiment does not load. It loads with only the instructions active, but not consent or metadata. So I am not sure how the code is even ending up in the results file I had before in the first place, since the consent chunk was deactivated for that run.

Thanks for any help!

Leo

[leaena]

Hi Jeremy,

He was not one of the people who were caught by our attention checking exclusion criteria. We discovered that he was using a translator because the code at the end of our experiment was an English word, and the code got translated, too, so it told him he had an invalid code, and he emailed me with screenshots to prove that he had completed the experiment that showed the code in Spanish. He also told me that the sentences had shown in English, but when I tested it with the same google translate extension I saw in his screenshots, they were in Spanish. I do worry that this has possibly been happening before without us catching it, since his performance on the task was actually above average…kind of a shame that we had to exclude him anyway.

I am interested in trying out the HTML canvas method, since that seems like it could be integrated with the PC template. But I’m not exactly sure how to get started – it seems like the PC HTML element is mostly for forms which are already created. Can I put “raw” HTML into an HTML element in PennController such that I can fill it with item.Sentence?

Thank you very much for your help!

Leo

[leaena]

Hi Jeremy,

Just FYI, using getVar("sentenceDisplayTime")._element.value also doesn’t work to give us the actual value (it shows up as the EventTime timestamp again, so that .set call just doesn’t do anything it seems?). We settled on ignoring it since it’s not a very reliable measure anyway given the existence of a replay button…

Thanks!
Leo

[leaena]

Hi Jeremy,

Should there always be two lines in the results file when you pass both “first” and “last”? If so, should there be a value somewhere in the line reporting “first” / “last” / “first-last” (in case only one selection happens) / “NA” (in case no selection happens)?

That would be my preference by default, since it’s not very convenient to delineate things in R by which line they appear on (and doubly so if the number of lines isn’t consistent). I don’t know how your code works but it seems like “first”/”last” etc. would be a good candidate to go in the area that just says “choice” now, since the particular choice on each selection is one of the choices that they made, meaning that “choice” is now ambiguous. I’m not sure how that would work with “all”, though, since you wouldn’t want to put in 14 lines for every scale if they selected on one of them 14 times…but in that case it might be enough to differentiate them by number. Since we end up with our data in wide format, the consistently two-line version is good for me, but I don’t know what other people use.

At first I thought I can’t really use the newScale strategy you suggested, since we want them to be able to replay the scene as many times as they want. But since we’re not collecting all of the selections, just two of them, would it be fine to just create a secondScale every time they press replay? Or would that cause errors with the name overwriting?

(Another side note: I don’t know if this is a one-or-two-off thing or a more generalized thing, but https://www.pcibex.net/wiki/selector-test-index/ this page’s text also doesn’t appear in “try it!”, even when I add print() to it.)

Thanks,
Leo

[leaena]

Hi Jeremy,

Thank you for the help!

Just a note: .settings.log(“first”, “last”) gives two lines in the results file per item, which are otherwise identical, having the same name. And if only one selection is made, it only produces one line per item instead of duplicating. This is a pain for data analysis and I haven’t figured out a way to deal with it post-hoc yet, so instead my current plan is to update a variable to the selected value by testing.selected each element in the scale.

(And a random side note: https://www.pcibex.net/wiki/button-wait/ the code here does not seem to produce the text in .settings.before() on the “try it!” page.)

Best,
Leo

[leaena]

Hooray, you’re the best! Thank you Jeremy! This makes my life much easier.

Best,
Leo

[leaena]

I’m not sure what I was doing wrong, since I thought I had tried that already, but it works! Thank you so very much!

Best,
Leo

[leaena]

Hi Jeremy,

This code does seem to work, or at least, I have it working here: http://spellout.net/ibexexps/leaena/test/experiment.html
However, the way that I understand to implement it is to copy the template and make one template for each type (a, b, c and d in my case). If I just have one template and name it “items” or something, PennController.Sequence does not find things named A or B or C or D and so it thinks there is nothing in the running order. However, when I copy the template 4 times, even though I use .filter() with each one so that it should only apply to one of the types, it repeats all of the items …twice? I am not sure if I’m doing something wrong or if it is not working the way it is supposed to. Probably the first. :]

Advice would be appreciated!
Best,
Leo

[leaena]

Thank you, Jeremy! I will see what I can make of this. (But may well have more questions later.)

Best,
Leo

[leaena]

Hi Jeremy,

Thank you for this information!

I am confused about the code you provided, though.
What are the buttons for?
What do the i and ii indicate?
How would we integrate this with a template? If the .csv file contains rows in which the conditions are typeA, typeB, and typeC, for instance? Where in this code does the template go? Or would we have to read in all of the types separately with separate templates?

Would this be a thing that would be labeled and then called in the normal ibex ShuffleSequence?
e.g. something like:
PennController.Sequence( randomizeExcludeSequence( anyOf(“typeA”, “typeB”, “typeC”) , “typeA”, “typeC” ) ).label(“allitems”)
where the ShuffleSequence goes seq(“instructions”, “practice”, “allitems”, “end_experiment”) or something?

Best,
Leo

[leaena]

Apologies – it was just missing an end ‘. Thank you! It works indeed!

[Author]

Posts