MouseTracker recording wrong coordinates

[irenecg]

Hi Jeremy,

We’re running an experiment in which participants have to make certain selections at the end of the trial, which we record using the MouseTracker element (and track the click coordinates for each time participants click). Since we are sending it to mobile phones, which have different screen sizes, at the very beginning of the study we ask participants to click at the corners of the screen (by putting a canvas on each corner that, unless touched, the study does not continue).

Our problem comes when we have seen that for a substantial number of participants their clicks coordinates fall outside the screen size. For example, if their screen is 360×720, they have a click recorded at x=350,y=1200.

We’ve double-checked that the screen sizes recorded are correct by taking their phone models and looking for the specifications and the
Viewport Sizes of each phone. Moreover, we’re videotaping the trials, so for some of these cases for which we have the wrong coordinates, we know that they were touching something specific on the screen (because they say so).

Do you have any idea of what could be going on?

Thanks a lot,

Irene

[Jeremy]

Hi Irene,

Do you use a MouseTracker element both for the clicks on the Canvas at the beginning of the experiment, and for the selections on the test trials? You can share your experiment with me by sending its URL at support@pcibex.net if you want

Did you make sure that the content of your experiment fits entirely on a 360×720 screen, and that your participants don’t need to scroll down to make a selection at the bottom of the page?

Jeremy

[irenecg]

Hi Jeremy,

We use a MouseTracker element at the beginning and another one for the test trials. And about the content of the experiment, it should fit, since it is printed in a canvas 90% width and 90% length of the screen size.

Thanks for the email address, I’m going to send you the link to the study 🙂

Irene

[Jeremy]

Hi Irene,

I’m sorry for not getting back to your sooner. Unfortunately, I’m still unable to identify the source of your problem. However, I realized something in the meantime, which is that smartphone users typically don’t connect a mouse to their device, but rather directly tap their screen: it makes me wonder how you manage to get coordinates at all for them, given that your script uses a MouseTracker element, which ultimately tracks mouse movements, not clicks (or taps, in the case of smartphones).

To add to the mystery, clientY (which is what the MouseTracker element reports) should never exceed the viewport’s height, which means that investigating possible scrolling-related issues is a dead end anyway.

I apologize for suggesting to use a MouseTracker element to track click coordinates. If you were to re-run your study, I’d now suggest you use a Function element to insert a plain javascript function that will better track clicks, like this:

newTrial(
    newCanvas("content", "50vw","50vh")
        .css("background-color", "orange")
        .print("center at 50vw", "middle at 50vh")
    ,
    newVar("clickx",[]).log(),newVar("clicky",[]).log()
    ,
    newFunction( ()=>$(".PennController-content").click( e=>{
       getVar("clickx").set( v=>[...v,e.clientX] )._runPromises();
       getVar("clicky").set( v=>[...v,e.clientY] )._runPromises();
    }) ).call()
    ,
    newSelector().add(getCanvas("content")).wait().wait()
    ,
    getVar("clickx").set( v=>v.join('.') ),
    getVar("clicky").set( v=>v.join('.') )
)

Let me know if you make any progress on this issue, I’d like to fix any related bugs in PennController to prevent it from happening again in the future

Jeremy

[irenecg]

Hi Jeremy,

What we did is to take where the mouse was (that is, mouse coordinates) every time a selection happened. Is that treatment problematic for touchscreen data? We have coordinates for most of them, but could this weird problem I told you about be related to this fact?

Irene

[Jeremy]

Hi Irene,

As it is now, the MouseTracker element only detects mouse movement, it does not detect mouse clicks at all. The content of the corresponding callback command is therefore run only when the cursor moves, and so your x/y variables are only updated on mouse movement, not on mouse click.

Which means that what you log upon selection is not technically the mouse coordinates when the click happens but the mouse coordinates resulting from the latest movement. Because there typically is no mouse movement whatsoever with touchscreen devices (unless a mouse is plugged in, of course), I don’t understand how your x/y variables can ever be instantiated at all. Maybe the slight finger swipe that can come with tapping the screen is interpreted as a mouse movement…

As I said, I don’t know what is happening exactly, so I don’t know how the different problems can be related. What I can say, however, is that the code from my previous message will effectively update clickx and clicky upon click, and no longer upon movement.

I’m sorry, I wish I could give your more insight into this problem

Jeremy

[Author]

Posts