Missing data (Failed submission)

[KenN]

Hi,

Recently I conducted an experiment in which 61 participants participated, and the Results panel shows “Preview of latest submission (out of 61)” indicating there were 61 participants, but when I downloaded the csv file, there were only 60 participants, with 1 missing.

I checked further into the results, and I found that the data of the 2nd participant is missing. The results.csv indicates that the presence of the participant in question because the following rows have been recorded:

#
# Results on Thu
# USER AGENT: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML
# Design number was non-random = 0
#
# Columns below this comment are as follows:
# 1. Results reception time.
# 2. MD5 hash of participant’s IP address.
# 3. Controller name.
# 4. Order number of item.
# 5. Inner element number.
# 6. Label.
# 7. Latin Square Group.
#

However, no further rows are recorded with this participant.

Now I remember that I had similar cases in the past. In all these cases, the participants whose data are missing seem to have completed the tasks to the end, because they provided us with the randomly generated codes that are shown at the end of the experiment.

My question is, how and under what conditions could this happen? Are there any precautions I can take to prevent this?

Demostration link:
https://farm.pcibex.net/r/ZGTPEq/

Thanks!

[Jeremy]

Hi,

Unfortunately that submission was sent to the farm’s servers precisely at the beginning of a short downtime episode, at 2:12am GMT on January 13. The submission entry was created, but the service crashed right before the data could be saved to memory

Unfortunately there is not much you can do on your end. I have been monitoring the problem more closely these past couple days, and the crashes appear to happen when there is a lot of incoming data at once, so minimizing the data in the submissions is good practice, but in your case, we’re talking about only 30 lines per submission, which is already very little

My attempts at controlling or working around those crashes have not been the most successful so far. The service is supposed to temporarily save a copy of the incoming data on disk before sending it to the database, but even that copy seems to have failed to be saved in your case

I apologize for the inconvenience. I am still actively working on addressing the issue

Jeremy

[KenN]

OK, thank you for the detailed explanation regarding the situation. I really appreciate the presence of this platform and also thank you for your efforts to improve the system!

-Ken N.

[merelo]

Hi,

I seem to have the same problem, but out of 10 participants, 6 were not saved. The details are the same; I have a few rows per “missing” participant recorded in my results file, like so:

# Results on Tue
# USER AGENT: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML
# Design number was non-random = 8
#
# Columns below this comment are as follows:
# 1. Results reception time.
# 2. MD5 hash of participant’s IP address.
# 3. Controller name.
# 4. Order number of item.
# 5. Inner element number.
# 6. Label.
# 7. Latin Square Group.

All 6 participants submitted between 15:57 GMT and 16:17:47 GMT; other participants’ data was successfully stored before and after this point. Can this be attributed to a downtime episode as well?

Would minimising the data in submissions help prevent this, and if so, how would I do that? (I still need to collect 160 participants.)

Thank you.
Best,
Merel

Demo: https://farm.pcibex.net/r/UeksSE/

[Jeremy]

Hi Merel,

The database shows that your experiment did receive 10 submissions, with between 1918 and 1922 rows for each of them, so you should see all your submissions and the corresponding rows in the results file. Maybe you tried to generate the results file before all the incoming data had finished being processed by the server?

Let me know if the problem persists

Jeremy

[merelo]

Hi Jeremy,

You’re right, thank you! I tried downloading it several times yesterday and it kept coming out the same way, but reloading it this morning made more rows appear.

[monicado]

Hi Jeremy,

I’m having a similar issue with missing results in my data. It appears I’m missing 4 submissions from the following experiment: https://farm.pcibex.net/r/SpzIlM/. I wonder if there is any chance this data can be recovered from the PCIbex server?

Thanks,
Monica

[Jeremy]

Hi Monica,

I see 13 submissions for that experiment, the four most recent ones received on April 27, April 28, May 6 and May 12. Each of your 13 submissions has between 54 and 58 rows

Were you expecting another 4 submissions (for a total of 17 submissions)?

Jeremy

[Author]

Posts