Different length of stored sound files with MediaRecorder()

[CogSciLanMethLab]

Hello Jeremy,

Thank you very much for all your support in programming these experiments in PCIbex!

In our study, we would like to collect several audio recordings from the participants in response to a picture stimulus presented on the computer screen. In short, the audio recording starts at the same time as the onset of picture presentation. After 2300 ms, the visual stimulus will be removed from the screen but the audio recording continues for 700 ms. That means that the total length of the sound files stored on the server should be 3000 ms long.

According to a post in the forum (see https://www.pcibex.net/forums/topic/timing-units/), the individual timer elements will usually not have the exact length as intended. However, the audio files elicited during our study exhibit a total length of either 3034 ms, 3054 ms or 3074 ms with a deviation of exactly 20 ms. We wonder why this is since the sound files are all collected during one trial block through a template. Even though there may be a delay, should the recordings not all be of the same length then? Is that a normal behaviour of PCIbex or the MediaRecorder() element, respectively? If so, could you please tell us how to avoid the deviation or at what position of time this deviation occurs – that will say, rather at the beginning or the end of the audio recording?

Moreover, when looking at the lines _End_ and _Start_ of the EventTime for the MediaRecorder() in the result file and then subtract the values, it should give us the total length between the start and the end of the audio recording (see https://www.pcibex.net/forums/topic/how-to-measure-the-total-time-of-experiment/). This time value, however, does usually not match the actual length of the sound files on the server which are all, as mentioned above, either 3034 ms, 3054 ms or 3074 ms long. Is there a way to use these values to solve the issue reported here?

The demonstration link for the study is: https://farm.pcibex.net/r/zBUnUG/
Please let us know, if you need more information!

I beg your pardon in case a similar question has already been answered!
Your help would be highly appreciated!

Best,
CogSciLanMethLab

[Jeremy]

Hi,

Unless you have high-quality hardware in a controlled environment, you won’t reach millisecond-precise computation. Every single instruction takes time to be executed on a machine, including low-level steps such as receiving and processing audio signal from the microphone or updating the display on the monitor, or higher level steps such as executing the PennController commands in your script

Some things work in a cyclic way, including the way Timer elements work, and I think audio processing too. If the process happens to start near the end a cycle and end near the beginning of another cycle, you might end up with two extra cycles (compared to a situation where the process happens to start at the beginning of a cycle and end at the end of the current cycle)

Because you call record on the MediaRecorder element and then wait on the two Timer elements before calling stop on the MediaRecorder element, the audios will be at least 3000ms long. But if the Timer elements happen to be longer than their respective duration, the audios will be longer too. Plus the delays due to internal processing

The differences you see between the End-Start subtractions and the actual audio file lengths are also due to processing delays: the MediaRecorder element logs when the record and stop commands are executed, but there also are small delays between when the commands are executed, and when they actually take effect at a lower level. Ideally, I should have designed the element to log both the timestamps of the record and stop commands, and the actual start/stop events of the recorder (which should be closer to when recording actually happens)

You might get closer to an actual duration of 3000ms by doing away with the extra layer of PennController and writing your experiment directly in javascript instead, but you will likely still end up with delays, because javascript is executed in the browser at a rather high level. If you really need high time accuracy, you’d be better off writing your experiment using a lower-level paradigm and running it on the same machine using a high-quality recording device for all your participants. Then you’ll get as close to 3000ms as can be nowadays

Jeremy

[CogSciLanMethLab]

Hello Jeremy,

Thank you very much for the fast reply and your detailed explanation on how to achieve a more precise time accuracy!

Based on your description, is it correct to assume that it is impossible to elicit audio recordings from the participants that are all of the exact same length?

Due to different configurations of the machine as well as the cyclic way certain elements and processes work, it is now clear that the recordings will not be exactly 3000 ms long. However, if each recording is elicited through the same template within one trial block, shouldn’t the respective sound files be of the same length even though they will effectively be a little longer than 3000 ms? Or is it the difference in additional time that will be added to the 3000 ms due to the aforementioned machine configurations, which causes the differing length of the audio files?

Thank you again for your help!

Best,
CogSciLanMethLab

[Jeremy]

Hello,

You could technically force recordings to all be 3000ms, for example by chopping them if necessary, or maybe even by writing a code optimized well enough that it would practically always produce a 3000ms audio file on most if not all configurations, but I don’t think that is feasible within PennController as it is currently written: there are just too many computational steps going on around and in parallel to the recording process

Even if the recordings are elicited through the same template within one trial block, you have no guarantee that the cycles will always align perfectly. Say your browser notifies your Timer elements every 20ms. What if one Timer element for one trials starts 10ms after the start of a cycle, but the same Timer element for the next trial starts 15ms after the start of a cycle? You’ll have a 5ms difference. And that’s assuming that performance is constant throughout the experiment, which is an unreasonable assumption. Performance will vary depending on what tasks the browser (and the OS more generally) processes in parallel, and even the simple fact that as the experiment progresses, the browser needs to maintain audio recordings in cache can impact performance (although, admittedly, it shouldn’t have a noticeable impact under normal circumstances)

That being said, since the script of your experiment calls log on all the relevant elements in the trial, you can get a pretty good idea of when the various trial events happen wrt the audio recording. The Start timestamp of the MediaRecorder will very closely coincide with position 0 of your audio recording (maybe a few ms off, but it’s unlikely it would reach 10ms, unless the browser is lacking processing power). Then the Print timestamp of the “training_pic” Image element will inform you when the image becomes visible on the page, so subtracting the Start timestamp of the MediaRecorder from it will give you a very good approximation of how many milliseconds in the audio recording that event took place. Same thing for the “fixcross_training” Image element

At the end of the day, as it is currently written, PennController won’t provide you with the means to conduct analyses of events wrt audio recordings with a precision to the millisecond. For most psycho-linguistic questions, a precision to the centisecond is usually sufficient, but I have to concede that this is a limitation of PennController at the moment

Jeremy

[CogSciLanMethLab]

Hello Jeremy,

Thank you again for all your help and in-depth description of the current limitations of PennController! This information will help a lot when it comes to designing our experiment and later analysis of the data.

Enjoy the upcoming weekend!

Best,
CogSciLanMethLab

[Author]

Posts