Sample trial: Picture Selection & Audio Playback

NOTE: this page assumes you are familiar with notions covered in this page.

Next sample trial: Rating scale & Input box

Setup

For this sample trial, you need to use the import method to retrieve the image and audio files we will be using.

Click on Update from git repo and enter the following URL: https://github.com/PennController/Tutorial.git and replace master with picture-audio in the branch box, then click the Sync button.
You should see a confirmation message appear below the button, reading Success: N files modified. If you don’t see the message, try clicking again. If it still does not work, reload your Ibex page and try again.

Warning: whenever you click on Sync, the content of the file main.js is erased and reverts back to the script in the Basics section below.

Basics

Let’s say we want to see whether people use plural on a verb to identify a referent early on. We will show our participants a pair of pictures with one or two fish, and ask them to indicate which is being described by the sentence they are listening to. The descriptive sentence is ingeniously designed to remain ambiguous until its last word, unless you notice the absence of an -s on the verb.

As a first step, let’s present the sentence as written text above the two pictures:

PennController.ResetPrefix(null);

PennController(
    newText("test sentence", "The fish swim in a tank which is perfectly round")
        .print()        // A test sentence
    ,
    newImage("competitor", "1fishSquareTank.png")
        .print()        // An image with 1 fish that swims in a square tank
    ,
    newImage("target", "2fishRoundTank.png")
        .print()        // An image with 2 fish that swim in a round tank
    ,
    newSelector("tank") // We indicate that target+competitor belong to a selection group
        .settings.add( getImage("target") , getImage("competitor") )
        .wait()         // On hold until target or competitor is selected
);

Size

Most people would probably need to scroll down to see the second picture, let alone click on it. We better reduce their size: we will use .settings.size(200, 200).

PennController.ResetPrefix(null);

PennController(
    newText("test sentence", "The fish swim in a tank which is perfectly round")
        .print()
    ,
    newImage("competitor", "1fishSquareTank.png")
        .settings.size(200,200)
        .print()
    ,
    newImage("target", "2fishRoundTank.png")
        .settings.size(200,200)
        .print()
    ,
    newSelector("tank")
        .settings.add( getImage("target") , getImage("competitor") )
        .wait()
);

Layout: canvas

It would be even better if the images appeared next to each other. We can place the images on a Canvas element to control their relative positions:

PennController.ResetPrefix(null);

PennController(
    newText("test sentence", "The fish swim in a tank which is perfectly round")
        .print()
    ,
    newImage("competitor", "1fishSquareTank.png")
        .settings.size(200,200) // Don't print it yet
        //.print()
    ,
    newImage("target", "2fishRoundTank.png")
        .settings.size(200,200) // Don't print it yet
        //.print()
    ,
    newCanvas("tanks", 500, 200)
        .settings.add(   0, 0, getImage("competitor") ) // 0 = left of canvas
        .settings.add( 300, 0, getImage("target") )  // 300 = 100px to the right of the right edge of competitor
        .print() // This prints the canvas, i.e. target and competitor side by side
    ,
    newSelector("tank")
        .settings.add( getImage("target") , getImage("competitor") )
        .wait()
);

You can add any type of (printable) element on canvas. Feel free to add some text below each picture (e.g. target and competitor, but only for your test runs of course!).

Audio playback

Now, the task really gets interesting with some audio instead of some text. At the Setup step, you imported an audio file containing a synthetic recording along with the images under chunk_includes. All we need to do is replace the Text element with an Audio element, like this:

PennController.ResetPrefix(null);

PennController(
    // newText("test sentence", "The fish swim in a tank which is perfectly round")
    //    .print()
    newAudio("test sentence", "fishRound.mp3")
        .play() // Immediately play the audio file
    ,
    newImage("competitor", "1fishSquareTank.png")
        .settings.size(200,200)
    ,
    newImage("target", "2fishRoundTank.png")
        .settings.size(200,200)
    ,
    newCanvas("tanks", 500, 200)
        .settings.add(   0, 0, getImage("competitor") )
        .settings.add( 300, 0, getImage("target") )
        .print()
    ,
    newSelector("tank")
        .settings.add( getImage("target") , getImage("competitor") )
        .wait()
);

Note: As of April 9, 2019, MP3 is the only audio format that is compatible with all major web browsers, but the Audio element also supports WAV and OGG files (that is, in browsers that support them).

Conditional feedback

You can print some feedback text after selection, telling participants whether they selected the right picture. You will need to use a Test command on the Selector element to verify the selection and execute commands accordingly. The script below illustrates such a case:

PennController.ResetPrefix(null);

PennController(
    newAudio("test sentence", "fishRound.mp3")
        .play()
    ,
    newImage("competitor", "1fishSquareTank.png")
        .settings.size(200,200)
    ,
    newImage("target", "2fishRoundTank.png")
        .settings.size(200,200)
    ,
    newCanvas("tanks", 500, 200)
        .settings.add(   0, 0, getImage("competitor") )
        .settings.add( 300, 0, getImage("target") )
        .print()
    ,
    newSelector("tank")
        .settings.once()
        .settings.add( getImage("target") , getImage("competitor") )
        .wait()
    ,
    newText("positive", "Good job!")  // Creation of a positive feedback text element
        .settings.color("green")  // Green for positive (don't print yet)
    ,
    newText("negative", "Oops, you were wrong...")   // Creation of a negative feedback text element
        .settings.color("red")    // Red for negative (don't print yet)
    ,
    getSelector("tank")           // Test whether the target image was selected
        .test.selected( getImage("target") )
        .success(
            getText("positive")   // Positive feedback if the test succeeds
                .print()
        )
        .failure(
            getText("negative")   // Negative feedback if the test fails
                .print()
        )
   ,
   newTimer("time to read", 2000) // Give some time to read the feedback
       .start()
       .wait()
);

Exercises

  • Some browsers won’t start playback unless you interacted with the page before: create a Button element (e.g. newButton("start", "Let's start!")) and tell the script to wait until a click happens before launching playback.
  • Resize images to 200x200px by default
  • Invite participants to press a key in order to validate the trial (tips: add a Text and a Key elements at the end)
  • Stop the audio at the end of the trial (action command stop) / Wait until playback is over before ending (action command wait)

Questions

The trial never ends if I use wait on the Audio element and click after playback is over. Is this a bug?

No. What happens is that you have two wait commands in your script: one for the Selector element first, one for the Audio element then. The first wait command, called on the Selector element, puts the execution on hold and releases it only after a selection happens. Only then is the second wait command, called on the Audio element, evaluated. If playback is over by that time, the wait command holds the execution forever, since it then only starts listening for end-of-playback after it already happened (though theoretically it could happen again, e.g. if you called print on the Audio element and the participant clicked and replayed the file). This is the default behavior of the wait command: it waits for the next (at the time of evaluation) end-of-playback event. You can tell it to check the first end-of-playback instead by passing "first" as its argument.

TL;DR: use .wait("first") instead of .wait()

I would like to select pictures by pressing keys. How do I do that?

Use the settings command keys on the Selector element, like this:

PennController.ResetPrefix(null);

PennController(
    newAudio("test sentence", "fishRound.mp3")
        .play() // Immediately play the audio file
    ,
    newImage("competitor", "1fishSquareTank.png")
        .settings.size(200,200)
    ,
    newImage("target", "2fishRoundTank.png")
        .settings.size(200,200)
    ,
    newCanvas("tanks", 500, 200)
        .settings.add(   0, 0, getImage("competitor") )
        .settings.add( 300, 0, getImage("target")     )
        .print()
    ,
    newSelector("tank")
        .settings.add( getImage("target") , getImage("competitor") )
        .settings.keys(            "F"    ,             "J"        )   // We associate each image with a key (spacing just for clarity, only order matters)
        .wait()
);

Additionally, you can disable clicks by calling .settings.disableClicks().

Can’t I just create the images when adding them to the canvas instead of having to create them first and refer back to them later?

Yes you can. Just try.

I added a request for a key press after selection, and the participant can now change their mind about the selection until they press the key. I don’t want that.

Try calling .settings.once() on the Selector element.

Do I really have to manually determine coordinates on canvas? It’s annoying!

You can use this page to create a draft of your canvas, and generate PennController-friendly code (you may have to adapt it a little bit though).

Next sample trial: Rating scale & Input box