Track Mouse Position

[irenecg]

Hi Jeremy,

I’m fairly new to this world so I’m not sure whether it is a request or just a general support question but… Is there a way to track the position of the mouse (versus just recording when something is clicked)?

Best,

Irene

[Jeremy]

Hi Irene,

Actually yes, there is a MouseTracker element in PCIbex, though it’s not officially documented. One concern with it is that it rapidly increases the size of the results file, as it reports a pair of numbers (the X and Y coordinates) every few milliseconds. Since you’re currently limited to 64MB on the PCIbex Farm, it practically imposes drastic limitations there.

I helped someone set up an experiment with the MouseTracker element on their own personal server and they were able to get good data, so if you are still interested and have access to a personal server, let me know and I can help you setting things up (it’s actually quite simple).

Jeremy

[vale.pevi01]

Hi Jeremy,
like Irene, I was wondering whether it is possible to track mouse position. If I understood correctly, the issue stands in the big amount of data that this tracking would generate. However, I’d be interested in getting the mouse coordinates at the mouse click (participant’s response). Is it possible / easier to be done?

Thank you a lot in advance!

Valeria

[Jeremy]

Hi Valeria,

Yes, getting the mouse’s coordinates upon click is actually easy enough, but you should make sure it’s really what you want. Keep in mind that different participants will have pages of different heights and widths. If you consider checking the mouse’s coordinates upon click, ask yourself whether it wouldn’t make more sense to place multiple clickable elements on your page, e.g. in the form of transparent canvases.

For example, this bit of script uses a MouseTracker element to keep track of the mouse’s coordinates and checks them later:

newVar("mousex"),newVar("mousey")
,
newMouseTracker("mouse")
    .callback( (x,y) => [getVar("mousex").set(x)._runPromises(),getVar("mousey").set(y)._runPromises()] )
    .start()
,
newCanvas( "wholescreen" , "100vw" , "100vh" ).print( "top at 0" , "left at 0" )
,
newSelector().add( getCanvas("wholescreen") ).wait()
,
getVar("mousex").test.is( v=>v<100 )
    .success( newText("The mouse is very much toward the left").print( 100 , getVar("mousey") , getCanvas("wholescreen") ) )
,
getVar("mousey").test.is( v=>v<100 )
    .success( newText("The mouse is very much toward the top").print( getVar("mousex") , 100 , getCanvas("wholescreen") ) )

But this other bit of script does almost the same thing more simply:

newCanvas("left", 100 , "100vh").print("left at 0", "top at 0"),
newCanvas("top", "100vw" , 100).print("left at 0", "top at 0")
,
newSelector()
    .add( getCanvas("left") )
    .callback( newText("The mouse is very much toward the left").print(100,"middle at 50vh") )
,
newSelector()
    .add( getCanvas("top") )
    .callback( newText("The mouse is very much toward the top").print("center at 50vw",100) )

Feel free to tell me more about how your experiment will use the mouse's coordinates

Jeremy

[irenecg]

Hi Jeremy,

Thanks a lot for your quick responses. I think I’ll try to adapt my experimental design to the solutions you proposed to Valeria, as they seem easier and lighter than what I had in mind.

Thanks,

Irene

[irenecg]

Hi Jeremy,

It’s me again. Sorry to come back to this message, but we realized we really need to track cursor position upon click. Our problem is that we need to record many clicks in the same trial, and, as far as we have seen, we are unable to record this into one variable AND log it for different values.

Let me explain: We have an experiment in which subjects have to click at different parts of the screen within the same trial. Ideally, these parts would be 2, but our subjects are children, so it is very possible that they click more than once, twice and even 10 different times to different parts of the screen (and we cannot predict where nor how many times). In that case, we would like to have the coordinates of the mouse for each of their clicks.
We tried:

newVar("mousex"),newVar("mousey")
,
newMouseTracker("mouse")
    .callback( (x,y) => [getVar("mousex").set(x)._runPromises(),getVar("mousey").set(y)._runPromises()] )
    .start()
,
newCanvas( "wholescreen" , "90vw" , "90vh" ).print( "top at 0" , "left at 0" )
,
newSelector().add( getCanvas("wholescreen") )
.callback(getVar('mousex').log(), getVar('mousex').log())
.log(all)
,
newButton('go!')
.size("10vw", "10vh")
.print("left at 90vw","top at 90vh")
.wait()

[Jeremy]

Hi Irene,

You will need a slightly different approach. Here is what I suggest: whenever there is a click (as captured by the Selector’s callback command) you append to two other Var elements the current values of the mousex/mousey Var elements (which are reset whenever the mouse moves, thanks to the MouseTracker’s callback command). Then you log the values of those two Var elements as a string of coordinates separated by periods (or any character for the matter, except a comma which is already used to separate fields in the CSV results file).

newTrial(
    newVar("mousex"),newVar("mousey"),
    newVar("clickx",[]),newVar("clicky",[])
    ,
    newMouseTracker("mouse")
        .callback( (x,y) => [getVar("mousex").set(x)._runPromises(),getVar("mousey").set(y)._runPromises()])
        .start()
    ,
    newCanvas("screen", "90vw","90vh").print("left at 0","top at 0"),
    newSelector().add(getCanvas("screen")).callback(
        getVar("clickx").set(v=>[...v,getVar("mousex").value]),
        getVar("clicky").set(v=>[...v,getVar("mousey").value])
    )
    ,
    newButton('go!')
        .size("10vw", "10vh")
        .print("left at 90vw","top at 90vh")
        .wait()
    ,
    getMouseTracker("mouse").stop()
    ,
    getVar("clickx").log().set(v=>v.join('.')),
    getVar("clicky").log().set(v=>v.join('.'))
)

Here’s an example of the two result lines that you get (I clicked three times, hence the three period-separated values):

1590592091,MD5HASHEDIP,PennController,0,0,unlabeled,NULL,Var,clickx,Final,149.576.632,1590592074542,Value at the end of the trial
1590592091,MD5HASHEDIP,PennController,0,0,unlabeled,NULL,Var,clicky,Final,276.329.321,1590592074542,Value at the end of the trial

Jeremy

[mwf2120]

Hi! I am relatively new to Pcibex and am trying to do the following: log coordinate location of images (between 2 and 5 depending on set randomization) that are dragged and dropped to another canvas. The figures can be placed anywhere and in any order on the new canvas. I’d really like to record where they place each figure. I’ve been able to implement the above approach and collect coordinate data, but the method fails when I try to drag and drop (probably because the images are “clicked” in one place and “unclicked” in another). Is there a better way to do this? Thanks so much!

Drag and drop sample approach:

newTrial("Set_2",
    newCanvas("figlist", "72vw","58vh")
        .css("padding","1em")
        .color("lightgray")
        .print("left at 5vw", "middle at 50vh")
    ,
    newCanvas("grid", "72vw", "58vh")
        .css("padding", "1em")
        .color("yellow")
        .print("left at 20vw", "middle at 50vh")
    ,
    newSelector("figures")
    ,
    newImage("man", "man_small.png").selector("figures").print(getCanvas("figlist")),
    newImage("boy", "boy_small.png").selector("figures").print(getCanvas("figlist"))
    ,
    getSelector("figures").shuffle()
        .log()
    ,
    newDragDrop("dd")
       .addDrag(getImage("man"))
       .addDrag(getImage("boy"))
       .addDrop(getCanvas("figlist"))
       .log()
    ,
    newTimer(5000).start()
    .wait()
)

Thanks for any help!

[Jeremy]

Hi,

You could add these three lines at the end of your script to get the coordinates of the visual elements at the end of the trial:

newVar("manXY").log().set(v=> JSON.stringify(getImage("man")._element.jQueryElement[0].getBoundingClientRect()).replace(/[,"{}]/g,'') ),
newVar("boyXY").log().set(v=> JSON.stringify(getImage("boy")._element.jQueryElement[0].getBoundingClientRect()).replace(/[,"{}]/g,'') ),
newVar("figlistXY").log().set(v=> JSON.stringify(getCanvas("figlist")._element.jQueryElement[0].getBoundingClientRect()).replace(/[,"{}]/g,'') )

Jeremy

[mwf2120]

Thank you so, so much! This worked.

Follow-up: is there a way to “center” the cursor while dragging and dropping so that when a person selects an image to drag, the cursor position defaults to a central position rather than participants being able to click anywhere on the image?

[Jeremy]

Hi,

You can’t control the cursor’s position, as that would pose serious security issues. You could, in theory, center the image at the cursor’s coordinates, but that’s not how I wrote the DragDrop element. What you could do to modify the behavior of the DragDrop element is upload PennElement_dragdrop.js to your project’s Modules folder, and look for these lines in the file:

start = {x: ev.clientX, y: ev.clientY, top: rect.top, left: rect.left, 
        old_top: ev.target.style.top, old_left: ev.target.style.left, old_position: ev.target.style.position};
$(ev.target).css({position: 'fixed', top: rect.top, left: rect.left});

and replace them with:

const newtop = ev.clientY - rect.height/2, newleft = ev.clientX - rect.width/2;
start = {x: ev.clientX, y: ev.clientY, top: newtop, left: newleft, 
        old_top: ev.target.style.top, old_left: ev.target.style.left, old_position: ev.target.style.position};
$(ev.target).css({position: 'fixed', top: newtop, left: newleft});

You’ll get a warning that the element type DragDrop is defined more than once when test-running the experiment, but other than that it should work as expected

Jeremy

[mwf2120]

Thank you — so grateful!

[Author]

Posts