Relevance & Research Question

There are two critical steps when setting up online panels with the exclusive reliance on mail invitations. The first one is the transition from the analogous invitation letter to a digitalized online questionnaire. Survey methods aim to minimize the effort for users and to increase the attractiveness and the benefits of a potential participation. However, nonresponse at the initial wave of a panel survey is not the only critical step to consider. The second one is the transition from initial wave participation to panel recruitment. Little is known about the potential enhancement of both transitions, from offline invitation to online participation and to panel recruitment by means of incentives. We investigate how mail based online panel recruitment can be facilitated through incentives.

Methods & Data

The analysis relies on two incentive experiments and their effects on panel recruitment and the intermediate participation in the recruitment survey. The experiments were implemented in the context of the German Emigration and Remigration Panel Study and encompass two samples of randomly sampled persons. Tested incentives include a conditional lottery, conditional monetary incentives, and the combination of unconditional money-in-hand with conditional monetary incentives. Furthermore, we assess the costs of panel recruitment per realized interview.

Results

Multivariate analyses indicate that low combined incentives (€5/€5) or, where unconditional disbursement is unfeasible, high conditional incentives (€20) are most effective in enhancing panel participation. In terms of demographic bias, low combined incentives (€5/€5) and €10 conditional incentives are the favored options. The budget options from the perspective of panel recruitment include the lottery and the €10 conditional incentive which break even at net sample sizes of 1,000.

Added Value

The key contribution of our research is a better understanding of how different forms of incentives facilitate a successful transition from postal mail invitation to online survey participation and panel recruitment.

Key words: Surveys, Online panels, Recruitment

Relevance & Research Question:

The recruitment stage is a key step in the set-up of a probability-based panel study, but it can also represent a substantial cost. A face-to-face recruitment approach in particular can be expensive, but a lower recruitment rate from a push-to-web approach risks introducing bias and putting a limit on what subsequent interventions to minimise non-response can achieve. This paper presents findings on using face-to-face and push-to-web recruitment approaches when recruiting to the NatCen Panel.

Methods & Data:

The NatCen Panel is recruited from participants in the British Social Attitudes survey (BSA). While normally conducted face-to-face, the 2020 BSA was conducted using a push-to-web approach in response to the Covid-19 pandemic. This study compares the recruitment rates and overall response rates of the face-to-face survey and push-to-web recruitment approaches. It also compares the demographic profile of panel survey participants recruited using each approach to explore to what extent any differences in recruitment and response rates translate into bias in the sample.

Results:

We find that, despite a higher recruitment rate and participation rate in panel surveys, the overall response rate using a push-to-web recruitment approach is substantially lower than when using a face-to-face recruitment approach due to lower response rates at the recruitment interview. There are also differences in the sample profile. For example, people recruited using a push-to-web approach were more likely to be younger, better-off financially, heavier internet users, and interested in politics.

Added Value:

Findings from this study will inform the future design of recruitment for panel studies, providing evidence on the likely trade-offs that will need to be made between costs and sample quality.

Relevance and Research Question

Underrepresentation of members of ethnic minority or immigrant-origin groups in most panels available to researchers hinders the study of these individuals experiences of daily life as well as racism and discrimination, or how these groups are affected by and react to important events.

Several approaches for reaching these groups exist but each method introduces biases. Onomastic classification is the current gold standard for identifying minority individuals; however, it is cost-intensive and has been shown to systematically miss well-integrated individuals. Respondent-driven sampling is increasingly popular for sampling rare or hidden individuals, while Facebook samples are the easiest and least expensive method to implement, but yield non-probability samples.

In order to identify the most efficient and representative methods of sampling and recruiting potential participants, we compare three different sampling methods with regard to the resulting biases in distributions.

Methods and Data

We compare three sampling methods:

(1) mail push-to-web recruitment of a probability sample with name-based (onomastic classification)

(2) web-based respondent-driven sampling (web-RDS)

(3) Facebook convenience sampling

In order to systematically test these methods against each other, we designed a set of experimental conditions. We test these conditions by sampling and recruiting a national sample of 1st-generation Portuguese migrants and their children.

We will compare the conditions based on factors which may affect recruitment into a national German online panel such as degree of integration, survey language and self-assesses language fluency, and income. Because we give individuals in the probability sample the option to respond via mail or web, we will additionally be able to compare differences across survey modes.

Results

We began fielding the probability sample condition at the beginning of March. We anticipate fielding of the additional conditions from April until June. This will allow us time to conduct analyses and develop preliminary results prior to the conference start date.

Added Value

Our paper will present an overview of our implementation of each method; our evaluation criteria; and preliminary results. We will provide a more realistic understanding of the potential biases, strengths, and weaknesses of each method, thus supporting researchers in making better informed methods choices.