Conference Agenda

Overview and details of the sessions of this conference. Please select a date or location to show only sessions at that day or location. Please select a single session for detailed view (with abstracts and downloads if available).

 
Session Overview
Session
A 2: Motivation and Participation
Time:
Thursday, 10/Sep/2020:
11:40 - 1:00

Session Chair: Bella Struminskaya, Utrecht University, Netherlands, The

Show help for 'Increase or decrease the abstract text size'
Presentations

Do previous survey experience and being motivated to participate by an incentive affect response quality? Evidence from the CRONOS panel

Hannah Schwarz1, Melanie Revilla1, Bella Struminskaya2

1Pompeu Fabra University (UPF), Spain; 2Utrecht University, The Netherlands

Relevance & Research Question:

As ever more surveys are being conducted, respondents recruited for a survey are more likely to already have previous survey experience. Furthermore, it becomes harder to convince individuals to participate in surveys and thus incentives are increasingly used. Both having previous survey experience and participating in surveys due to incentives have been discussed in terms of their links with response quality. In both cases, theoretical arguments exist that argue these factors could increase or decrease response quality. Empirical evidence is scarce and findings are mixed. This study thus aims to shed more light on the link of previous survey experience and participating due to incentives with response quality.

Methods & Data:

We analysed data of the probability-based CROss-National Online Survey (CRONOS) panel covering Estonia, Slovenia and Great Britain. We use three response quality indicators (item nonresponse, occurrence of primacy effects and nondifferentiation) as outcome variables and indicators for having previous web survey experience and being motivated to participate due to the incentive as predictors of main interest in our regression models.

Results:

We found that previous web survey experience had no impact on item nonresponse and occurrence of a primacy effect but reduced nondifferentiation. Being motivated to participate by the incentive did not have a significant impact on any of the three response quality indicators. Hence, overall we find little evidence that response quality is impacted by either of the two factors, previous web survey experience and participating due to the incentive, which are increasingly present in target populations these days.

Added Value:

We add to the scarce pool of empirical evidence on the link between having previous survey experience and participating in surveys due to incentives with response quality. An explicit measure of extrinsic motivation is used in contrast to previous research which often simply assumes extrinsic participation motivation to play a role if incentives are provided.



Moderators of response rates in psychological online surveys over time. A meta-analysis

Tanja Burgard1, Nadine Wedderhoff1,2, Michael Bosnjak1,2

1ZPID - Leibniz Institute for Psychology Information, Germany; 2University of Trier, Germany

Relevance & Research Question:

Response rates in surveys have been declining in various disciplines in the last decades. Online surveys have become more popular due to their fast and easy implementation, but they are especially prone to low response rates. At the same time, the increased use of the internet may also lead to a higher acceptance of online surveys in the course of time. Thus, the overall time trend in response rates of psychological online surveys is in question. We hypothesize a decrease in response rates and examine possible moderators of this time effect, as the invitation mode or contact protocols.

Methods & Data:

We searched PsycInfo and PubPsych with the search terms: (Online Survey or Web survey or Internet survey or email survey or electronic survey) and (response rate or nonresponse rate). This resulted in 913 hits. The abstracts of these records were screened to exclude studies not reporting results of online surveys. We excluded 84 records, resulting in 829 articles for full text screening.

So far, 318 full texts have been screened to assess, whether an article reports response rates of online surveys only and if the information on the participant flow is sufficient to compute response rates, the outcome of interest. Using the metafor package in R, mixed effects multilevel models will be used to investigate the hypothesized time effect and the moderating effects of recruitment, invitation and contact protocols.

Results:

With information on 95 samples from 83 reports, the estimated mean response rate in this meta-analysis is 46 %. There is evidence of declining response rates over time. Personal contact to invite respondents improves the willingness to participate. The response rates are slightly higher in samples that received a pre-notification. The number of reminders has no effect on response rates.

Added Value:

The results of the meta-analysis are important to guide decisions on the conduction of online surveys in psychology. To reach the target population, personal contact and use of a pre-notification is recommended. Further investigations, for example on the use of incentives and on the effect of respondent burden, will follow.



We’re only in it for the money: are incentives enough to compensate poor motivation?

Valentin Brunel, Blazej Palat

Sciences Po, France

Relevance & Research Question:

Attrition has been one of the main targets of survey research in panels from its beginnings (Lazarsfeld 1940, Massey and Tourangeau 2013).

This research aims at using different tools measuring motivation (closed items, paradata, open-ended questions…) to understand how initial motivation, interacting with different types of incentives, plays a part in the decision to leave the panel in the context of changing panel functioning.

Methods & Data:

ELIPSS panel, whose panelists were equipped for survey completion with dedicated tablets connected to the internet, was active from 2012 to 2019. Panelists’ motivations to join the panel were systematically measured during recruitment. Their response quality and motivation were also assessed using paradata and recurrent survey measures. As the panel’s functioning was about to change, an experiment on using unconditional differential incentives to encourage staying in the panel was designed. Three randomly selected groups of panelists were formed. The first received financial incentives of the same amount repeatedly: at t1 and four months later at t2. The second received the same financial incentive once at t2, and the third received the same financial incentive coupled with a gift at t2. We analysed the influence of those incentives on panel attrition in interaction with motivation indicators.

Results:

Initial motivation studies in the ELIPSS panel have outlined the importance of this indicator in further behavior inside the panel. We observed that as panelists declared themselves more interested by incentives, their chances of leaving increased. Results of the experiment show a marginaly significant effect of incentive type on attrition. Panelists who were given additional incentives didn't seem to remain more often, quite the contrary. However, interpretation of those effects should be clearer when taking motivation into account. Certain types of incentives may have differential results on different types of panelists.

Added Value:

The study results add to the knowledge of how the effects of initial motivation to join an online, non commercial research panel interact with unconditional differential incentives to influence panel attrition in an exceptional context. It is thus of primary importance for panel management purposes.



Should I stay or should I go? - Why do participants remain active in market research communities?

Ruth Anna Wakenhut, Jaqueline Fürwitt, Sophie Vogt

KERNWERT, Germany

Relevance & Research Question:

The conception and realization of medium- and long-term market research online communities confront researchers with great challenges: In particular qualitative communities are dependent on committed participants staying during the whole field time in order to gain insightful, relevant answers. Some community projects seem to fail precisely in keeping people motivated over a longer period of time.

Through other studies we have already learned that monetary incentive plays an important, but not all determining role. What other factors are relevant? We want to explore the reasons for commitment in research communities and how it can be positively influenced to achieve comprehensive participation throughout a longer community. We focus on the field time, on everything happening after recruitment. We want to better understand how a good online community works from the participants' perspective.

Methods & Data:

We apply a mixed approach and conduct our research in 3 phases with increasing depth: A quantitative questionnaire with about 200 participants is followed by a short digital qualitative project (e.g. forum, associative and projective tasks) with about 40 participants and 8-12 one-hour webcam interviews. This iterative approach allows a gradual identification of participants' experiences, expectations and needs in relation to online communities. We work with partici-pants from different recruitment sources (panel and field service providers), all already took part in qualitative digital research. The total field time is about 2 weeks.

Results:

None yet. We will conduct the study in January-February 2020 and will be able to present the results at the conference.

Added Value:

Our industry is facing the challenge of continuing to attract people who take the time to answer our questions – even over a longer period of time. Especially in qualitative research communities we need open dialogue partners who are willing to stay a while with us. Without participants, there is no market research. Motivated participants who feel valued are likely to provide personal, authentic answers. Understanding why people stay in online communities is important to ensure that those needs are met and people continue their participation.