Missing data rows in the results file

[ozgebakay]

Hi,

I collected SPR data from 80 participants. When I downloaded the results, 11 participants’ data rows are not saved, but I see the following only:

# Results on Sun 28 Nov 2021 12:33:27 GMT
# USER AGENT: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/96.0.4664.45 Safari/537.36
# Design number was non-random = 13
#
# Columns below this comment are as follows:
# 1. Results reception time.
# 2. MD5 hash of participant’s IP address.
# 3. Controller name.
# 4. Order number of item.
# 5. Inner element number.
# 6. Label.
# 7. Latin Square Group.

9 of these participants are before the very last participant, and the other 2 are somewhere in the middle.

I have data for the rest of the participants, so I am thinking these participants’ data may not be saved because of space limitations.

Also, when I download the data, I get a warning saying something like “max row, some data may be missing”. Is there a way to get these missing data? Could they be saved somewhere else?

Best,
Özge

[Jeremy]

Hi,

The message informs you that your project contains more than 100,000 results lines, and that you will only be served results files up to 100,000 lines. The solution is to download the results in multiple batches, only including series of submissions that total fewer than 100,000 results lines at a time (refresh the page after you download a batch, as sometimes memory gets clogged). Then you can concatenate the multiple CSV files that you’ll have downloaded into a single one on your local machine if you like (on Unix systems, you can do it very quickly with a command like cat results*.csv > results_all.csv)

I recommend you unpublish your experiment when you do that, to avoid missing new incoming data in the process, and once all your results have securely been downloaded to a safe location, delete them from the farm to free up space in our database, to make saving and accessing future data faster, and to avoid facing this situation again. (Remember to publish the experiment again if you plan on collecting more data)

Jeremy

[Author]

Posts