Reply To: High number for results of project

PennController for IBEX Forums Support High number for results of project Reply To: High number for results of project

#7317
Jeremy
Keymaster

Hi Noelia,

The PCIbex Farm will only let you generate results files with 100k lines max (excluding comments lines) so my recommendation is you estimate how many participants fit in 100k lines, and you download the file and clear the results at those intervals. Just make sure you pause data collection in advance (turn off “published” on your project’s page) so you don’t get new data caught in while deleting results from the farm. Of course you can always not do that and keep collecting data, and use the “only include submissions” option to download your results in multiple batches of less than 100k lines each (maybe refresh the page each time too, if you do that)

The issue with large results files is that, the more lines the PCIbex Farm has in its database, the longer it takes to access it, resulting in longer delays for saving future incoming results and for downloading results files. We have had several crash episodes in the past related to large results files, so we try to encourage users to keep them light by limiting the number of lines logged, downloading the files regularly and deleting them once safely stored on a local device

We won’t initiate deletion of results without asking for consent. However, technical problems might cause data loss, for example if a participant loses their connection while taking your experiment, or if the server crashes or is temporarily down at the moment they send their responses (which, again, is more likely for larger data transactions, because of the memory load they incur)

Jeremy