Testing in the lab versus online testing

Intro

Testing in the lab has a number of distinct advantages over online testing through video conferencing. For example, the researcher has more control over the testing situation, and digital literacy is not necessarily required from participants. A disadvantage, however, is that participants may find that the threshold of traveling to the lab to participate is too high, due to time constraints or mobility issues, inadvertently leading to a smaller pool of participants. Luckily, during the COVID-19 pandemic, the overall experience was that online data collection through video conferencing was feasible overall (e.g., Zhao et al. 2020; van der Ploeg & Blankinship 2022; van der Ploeg, Lowie, and Keijzer 2023; Nijmeijer et al. in preparation; Brouwer et al. forthcoming).

Benefits and drawbacks of online testing

Benefits of online data collection include that participants can be tested from their own home, saving them travel time and also often making them feel more comfortable. Of course, online data collection has some drawbacks, too: participants may not be proficient in using video conferencing tools, may not have a stable internet connection, or do not own a webcam and/or microphone. Issues like this can be circumvented by sending basic instructions (both in written form and using videos) regarding the use of video conferencing software and recommending the participant to practice setting up the software before the testing session takes place.

Practical considerations

1. Software
   One should consider the benefits and drawbacks of different types of available video conferencing software packages (van der Ploeg & Blankinship 2022). For example, Google Meet is relatively easy to use, and does not require one to download software or create an account, like Zoom does. However, if one’s university does not have a privacy agreement with Google, using it may not adhere to (the institution’s) privacy guidelines

2. Practicing video conferencing before data collection
   Conducting meetings prior to the actual data collection (e.g., the screening session) through video conferencing also gives both the participant and the researcher a chance to address potential technical issues. This decreases the odds of problems when research data is collected.

3. Computers vs tablets . Older participants often do not have access to a desktop computer or laptop, or they are more comfortable completing tasks on their tablet (e.g., iPad) than on a computer or laptop. In these cases, it may be advisable to program a touchscreen-friendly version of tasks (cf. Van Der Ploeg and Blankinship 2022).

4. Access to the materials. Although we do live in an increasingly technology-focused society, researchers should not automatically assume that all potential participants, particularly those from a lower socioeconomic background, have reliable or consistent access to a computer, email, or smartphone. Therefore, if researchers want a representative sample (i.e., one that does not lead from the overrepresentation of highly educated and high socio-economic status people), online data-collection may not be suitable.

5. Cheating. Something that may seem unexpected is that participants in studies that collect data via video conferencing may be more prone to cheating (e.g., writing down answers on a working memory task). Potentially, this is because they feel a pressure to 'perform' well. Therefore, it is helpful to remind participants that the goal of the test is not to get a perfect score, since many tasks are designed to prevent ceiling effects.

References

Further reading