Conference Agenda

When conducting mobile-phone based assessment in people’s daily life, researchers need to know how design characteristics (e.g., study duration, sampling frequency) affect selectivity and compliance, that is, who will participate in the study and provide how much information. We addressed the issue of selectivity in the Innovation Sample of the Socio-Economic Panel, who were invited to participate in an ESM study on happiness, and were either offered only feedback in 2015 or feedback and monetary reimbursement in 2016. Participation increased from 7% in 2015 to 36% when receiving feedback and monetary reimbursement, and compliance was much higher as well (29% in 2015 vs. 86% in 2016). Furthermore, participants differed from non-participants regarding age and gender, but negligibly regarding personality characteristics. To further examine design effects on participants’ compliance, we conducted a meta-analysis on ESM studies from 1987 to 2018, from which we coded a random subsample of 402 studies regarding sample characteristics, study design (e.g., study duration, sampling type and frequency, sensor usage), type of incentives, as well as compliance and drop out. Initial results showed that associations between design characteristics and compliance varied with sample type and type of incentive. For example, in adolescent and young adult samples, compliance was non-linearly related to number of assessments, whereas in adult samples compliance was larger with larger numbers of assessments. Providing incentives to participants, especially monetary incentives, predicted higher compliance rates compared to no incentivizing, except in physically-ill samples. This latter effect is likely attributable to high intrinsic motivation to provide information among participants dealing with chronic and other illnesses. We thus conclude that both the study design and the incentive should be adapted to the intended sample and we offer first empirical findings to guide these decisions.

Relevance & Research Question: Within the survey context, geofences can be defined as geographical spaces that trigger a survey invitation, when an individual enters, leaves, or stays within this geographical space for a prespecified amount of time. Geofences may be used to administer context specific surveys, e.g., an evaluation survey of a shopping experience in a specific retail location. While geofencing is already used in other contexts (e.g., marketing and retail), this technology seems so far underutilized in survey research. In this talk, we will share our experiences with the implementation of geofences within an app data collection study. Given the limited research on this topic, we will answer the following exploratory research questions: How well did the geofencing approach work? What are the reasons for geofencing to fail?

Methods & Data: In 2018, we invited participants of the PASS panel survey to download the IAB-SMART app. The app passively collected smartphone sensor data (e.g., geolocation, app usage and location) and administered short surveys. Overall, 687 individuals installed the app. While most in-app surveys were triggered by a predefined time schedule, one survey module was triggered by a geofence. To define geofences and trigger survey invitations our app used the Google Geofence API.

Results: Overall, the app sent 230 invitations and received 225 responses from 104 participants. However, only 56 of the 225 responses stated that they actually accessed the geofence. Cross-validating the Google Geofence API survey triggers with our custom built geolocation measurement in the app shows frequent mismatches between the two. Our data indicates that in many cases individuals should not have received a survey invitation because they were actually not in the specified geofence.

Added Value: Existing literature about geofencing (largely consisting of Youtube videos and short blog posts) only provides a short introduction to this technology and virtually no use of geofencing is documented in survey research. Our presentation evaluates the reliability of geofences, shares lessons learned, and discusses limitations of the geofencing approach to a broader audience.

Relevance

At the beginning of the era of online surveys, these were programmed to be answered using desktop PCs or notebooks. However, due to technical development such as the increasing role of mobile devices, studies on online survey research detect an increase of questionnaires that are answered on mobile devices (md). However, survey navigation on md is different compared to PC: it takes place on a smaller screen and usually involves a touch pad rather than a mouse and a keyboard. Due to these differences in questionnaire navigation, some of the traditional used web question formats are no longer convenient to be answered on a md. The most common formats are matrix questions. To deal with this development, so called mobile-friendly or responsive-designs were developed, which change the layout of specific questions that are not convenient on a md into an alternative mobile-friendly-design. In case of matrixes, these were separated into item-by-item questions which are suggested to be more comfortable to answer on a mobile device.

Research question

However, from a psychometric perspective the question whether these changes in question format produce comparable results is too often ignored. Therefore, this paper elucidates the following research question: Do different versions of responsive-designs actually produce equivalent response?

Data & Methods

Using the data of the first two waves of the Germ and Emigration and Remigration Panelstudy we can base our analysis on more than 19.000 cases (appox. 7.000 using different md). As GERPS makes use of a responsive design, we are able to investigate measurement invariance between different md and desktop device groups.

Results:

As the data management is still in progress and will be finished in the end of October, we will be able to present first-hand information based on fresh data. However, first initial analyses reveal differences between md and desktop device versions.

Added Value

Our study is one of the first that elucidates the equivalence of responsive design options. Thus, it enhances the perspective on the existence of possible new biases and error sources due to the increased use of md within web surveys.

Relevance & research question:

Traditionally, public opinion (PO) has been measured through probability-based surveys, which are considered the gold standard for generalising findings due to their population representativeness. The turn to social media data (SMD) to gauge PO in recent years has however led to a discussion about the potential for augmenting or even replacing survey data. Survey and social media researchers have, therefore, increasingly explored ways in which social media and survey data are likely to yield similar conclusions. In this context, two core challenges can be identified: i) researchers have mainly emphasised on replacement of survey data by SMD, rather than on the complementarity between both data sources; ii) there are currently two understandings of PO, which makes complementarity of both data sources quite difficult. As a result, researchers still need more guidance on to best complement SMD with survey data.

(Methods & Data):

Whereas the recent extension of the Total Survey Error framework to SMD is an important step to account for the quality of both data sources, we would like to propose an addition step that should come ideally before the discussion and evaluation of the quality of the collected data. Building on four key challenges, we develop a reflexion matrix to provide practical guidelines dealing with the complementarity of both data sources for the study of PO.

Results:

Our results convey two main take-home messages: i) we demonstrate that the main approach validating what we have found via surveys using SMD is problematic as survey measures convey an idea of simplicity and aggregation, whereas SMD are complex and multi-dimensional; ii) we provide researcher with an orientation of how SMD can be a potential complementary source to survey data.

Added Value:

We argue for the necessity to develop different and complementary views of PO if conducting research with a mixed-method approach, where complementarity of the data sources is one essential criteria. In addition, we point to possible solutions from other disciplines which have been little considered in studies of PO yet.

Relevance & Research Question: In many countries and contexts, survey researchers are facing decreasing response rates and increasing survey costs. Data collection is even more complex and expensive when rare or hard-to-reach populations are to be sampled and surveyed. Alternative sampling and recruiting approaches are usually needed in these cases including non-probability and online convenience sampling. A rather novel approach to recruit rare populations for online & mobile survey participation uses advertisements on social media. In this study, I present the fieldwork results of a survey of Lesbian, Gay, Bisexual, Trans, and Queer (LGBTQ), for which participants were recruited via ads on Facebook and Instagram.

Methods & Data: In 2019, an ad campaign was launched on Facebook and Instagram to recruit German web survey participants self-identifying as LGBT or Q. The survey was part of a research project on LGBTQ and Rainbow Families in Germany conducted at Bielefeld University. The questionnaire covered a variety of topics including partnership, family/children, employment, health, and experience of discrimination.

Results: Over the course of 5 weeks, over 7,000 respondents participated in the web survey (completed interviews). Comparatively few screen-outs and survey break-offs were observed throughout the fieldwork period. Plausibility checks and measurement error indicators point to good data quality.

Added Value: This study provides novel insights on how to plan and conduct an ad campaign for recruiting rare and hard-to-reach populations for web survey research. First, the practical details and challenges of fieldwork and campaign management are discussed. Second, the survey data is analyzed focusing on survey error and potential data quality issues. And finally, the social media sample survey is compared to a probability-based, face-to-face sample survey of LGBTQ in Germany that was carried out simultaneously as part of the research project.

Relevance & Research Question:

French people have discovered a passion for recycling! Circular economy is growing, offline with “brocantes” (=flea markets) everywhere, and online with online marketplaces selling second hand products.

But French people do not use ebay, they use LeBoncoin.fr; the 4th most visited website in France (with 26 million unique visitors per month, right below giant tech companies) where all types of secondhand products can be found.

Another surprise comes from the youngsters (aged 15-25 yo). They do love old stuff! For many categories of products, ranging from clothes to smartphones, younger consumers would rather shop second hand than new.

This research, conducted for Le Bon Coin, is aimed at explaining the attractiveness of old products to the younger generation.

Method

This research relies on a two-step process

1) Online behaviour (web visits and apps usages) of 300 young French tracked for more than one year. We have focused on their secondhand products shopping online patterns.

2) These respondents were then invited to participate in an online community. We organized a 7-day community on the topics of consumption, collaborative consumption and online vintage shopping with more than 100 individuals.

Results

This research shows second hand goods fulfill two major aspirations of these new generations; firstly the quest for more sustainable practices, and secondly the desire for individual distinction. Circular economy is perceived as ethically more acceptable, and second hand products tend to be more unique or exclusive: vintage goods offer double the reward for these young consumers, who wish to appear more responsible and less conformist than their elders.

These results are based on a segmentation of young people buying old stuff online.

Added value

These findings help to understand the disruptive mindset of these new customers and will be of great interest to retailers and brands that need to quickly address shifts in attitudes and behaviors of young consumers. From a more actionable perspective, our research also focuses on best practices for online platforms for buying, selling and trading used products to highlight what leboncoin has achieved in France and what ebay did not.

Relevance & Research Question:

Attitudes towards data privacy, either with regards to one’s own data or data more broadly, has the potential to be one of the major factors in the functioning of commercial markets and the effective operation of government and public policy. However, there are legitimate concerns about that data being used in ways that lead to negative outcomes for us as individuals, for groups that we identify with, or for society more broadly. Governments are continuously attempting to balance those costs and benefits, and commercial organisations need to be wary of losing their social licence with regards to data. Public attitudes towards data privacy and data governance is a key input into this balancing, and there is a growing social science literature on how these attitudes are shaped, how they vary between and within populations, and the impact they have on decision making. This paper consolidates existing literature and presents new empirical research (observational and experimental) with the aim of developing a theoretical model of data privacy. This predictive model builds on the behavioural economics literature and combines elements of risk, trust, perceived benefits, perceived costs, and behavioural biases.

Methods & Data:

The new data presented in this paper is based on a series of survey questions (experimental and attitudinal) fielded on the Life in Australia panel. Primary data collection is supported by analysis of survey data from comparable jurisdictions (primarily New Zealand, the US and Europe)

Results:

The main finding is that the model is shown to predict behaviour with regards to data linkage consent, public health data, and the consumer data right/open banking legislative reforms. Specifically, we show the importance of trust in government, the interaction that trust has with risk preference, the complicated impact of framing, and the importance of issue salience. We also demonstrate the instability of preferences through time.

Added Value:

While primarily based on Australian data, this is the first theoretical model of data privacy that (a) i built on experimental data from a representative panel (b) utilises behavioural insights and biases and (c) is shown to predict actual behaviour.

Relevance & Research Question: Passively measured digital trace data (e.g., from social media platforms or web browsing logs) are increasingly used in the social sciences as a cost-efficient alternative or as a complement to surveys. In this context, particularly augmenting survey data with information from respondents' online activities represents a promising direction to utilize the advantages of both worlds. However, extracting meaningful measurements from digital traces for substantive research is a challenging task. In this study, we use natural language processing techniques (NLP) to classify respondents based on the news content they consumed online and study the link between individuals’ political preferences and their online behavior.

Methods & Data: We augment survey data with web browsing logs by members of an online panel running in three countries (France, Germany and the UK). Data were collected during four weeks in May 2019, covering the European elections. Respondents answered questions about their voting behavior and political preferences in two surveys, one before and one after the election. In addition, respondents gave consent to having their online activities on their personal computers and mobile devices recorded. We extract information about news content consumed by respondents and search queries made to online search engines from the web logs. To analyze this content, we use natural language processing techniques (BERT; Bidirectional Encoder Representations from Transformers). We then model respondents’ political preferences and voting behavior using features from the content consumed online.

Results: Preliminary results indicate that the content of news consumed is a good predictor of voting behavior and political preferences and outperforms information that merely summarize visits to news and other websites. Overall, however, records of online activities do not seem to predict voting behavior and political preferences in an almost deterministic way.

Added Value: Our results add to the debate of selective news exposure and changes in political engagement due to Internet use. Moreover, they confirm previous findings regarding the limited effect size of internet use and selective news exposure on political behaviors and preferences. Overall, it seems that online societies may not be as fragmented as some early commentators postulated.

Relevance & Research Question: Global climate change continues to increase in visibility through observable environmental events, news, or political attention. As a result, psychological and mental health implications have begun to be considered in academic and practitioner circles. Awareness of the threat of climate change has led to an increase in “eco-anxiety” or “climate anxiety” in which individuals feel stressed, worried, nervous, or distressed about the future on Earth. This research aims to add to the growing body of literature on climate anxiety by examining American-based social media posts with the intention of understanding what type of climate-related events, news stories or political activities impact expressions of climate anxiety. Additionally, this research seeks to examine if people suggest taking any household, local, or national actions in light of climate anxiety, such as reducing consumption, switching to green products, or advocating and voting for climate friendly politicians.

Methods & Data: With access to Twitter and Reddit data through Brandwatch Consumer Research software, close to 50,000 posts about climate anxiety were collected from January 2016 through October 2019 (45 months) geolocated in the United States. An exploratory data-led approach will be taken with a sample of this data to establish themes and actions being discussed. Once these themes and actions are established, automated text analysis will be used to populate the categories against data from the full dataset such that they will continue to segment new conversation as it happens. This will allow revisiting the data in the future to become aware of any longitudinal changes in people’s opinions and behaviour.

Results: The analysis will be completed early 2020.

Added Value: The research intends to create a framework for understanding how social data can be used to examine drivers of climate anxiety, as well as possible tipping points that create impetus for personal, social, or political action among those most worried about the impact of climate change on our future. Additionally, understanding tipping points can aid pro-climate politicians and activists to capitalise on events and stories that most trigger action.

Have you ever conducted innovation research online and found yourself in a situation where you don’t entirely trust what consumer feedback is telling you? Many of us have had to deal with overstated interest in a new product. Uncovering both rational and emotional needs is vital for new product development (NPD) strategies to accurately size the unmet need or opportunity. While traditional qual techniques can uncover emotions, the results can’t easily be scaled. Alternatively, quant research can deliver stated emotions, but lack depth.

Together with Johnson & Johnson, we developed a new hybrid Qual-Quant-AI online research approach for early-stage NPD research. By using a voice analytics tool, we can analyze ‘how’ people communicate their needs, attitudes and interest, to better uncover emotions for more effective innovation strategies. The voice analytics AI tool we selected, audEERING, can detect emotions from voice. In an online survey with voice input a hypothetical ‘Smart Health Band’ was evaluated by consumers. We 1) analyzed the content of responses to look at what people said and 2) using the voice AI tool analyzed how they said it and which emotions were present.

The content analysis - the what was said - showed high interest in the concept. However, the voice AI delivered additional, unexpected results: while still high, the implicit or subconscious interest was not as high as the content analysis had suggested. Analyzing the emotions delivered through voice showed a more realistic level of interest in the product.

Especially for new products it is essential to get accurate estimates for the product’s market potential that are not overstated. The additional voice element helped Johnson & Johnson calibrate the interest in the product to show how much of it was genuine, and in consequence make better informed choices about future strategy. In addition, the emotional analysis uncovered differences in gender reactions. These segmentation insights could prove valuable for future marketing and communications strategies.

Relevance & Research Question:

The benefits of using VR prototypes for higher-cost categories in innovation and design projects are well known – logistics, greater modification flexibility, lower cost, virtual in-store and competitive choice scenarios.

With VR technology becoming more sophisticated and user-friendly, how can market research further benefit from it?

We will present a case study on a new fridge-freezer concept with Electrolux and a leading VR specialist company.

Methods & Data:

54 VR explorations across three countries - Germany, Italy, Sweden - accompanied by a total of 9 in-depth focus groups.

Results:

Across all age groups the VR part worked very well, both as a stand-alone and in combination with focus groups. Following outputs were key benefits:

Usability: Detailed feedback was gathered – particularly in comparison to fragile 3-D renderings.

Immersive: With prototypes seeming so real, we received more spontaneous and little post-rationalizing feedback.

Curiosity: A thrilling tech factor led to higher engagement.

Involvement: Fridges – a potentially lower interest category – enjoyed higher levels of excitement.

Playfulness: Tasks were treated more like games.

Flexibility & Speed: 3D renderings could be changed from one fieldwork session to the next.

Democracy: VR created equal conditions for prototype and comparison device.

Realistic Environment: Simulating a realistic shop floor atmosphere including competitive products was made possible.

Focus Groups + VR:

Higher participant focus due to the VR experience– everyone was highly engaged.

Higher attention to detail due to the amount of time spent with the prototypes during the VR experience!

There was nevertheless a strong creative dynamic present in the F2F groups.

Research moderation expertise is needed to manage “digital excitement” –managing overexcitement that leads them to jumping from one design aspect to the next.

Added Value:

The VR approach was more cost-efficient, more environmentally friendly, offered higher flexibility and in the end enabled deeper and more valuable insights – particularly with regards to usability!

Benefits of VR continued after the research: designers were more open to implementing the design changes as they felt less "emotionally attached" to the stimulus (compared to physical models). Our results were thus received more openly and applied without hesitation.

Relevance & Research Question: For large companies it has become increasingly difficult to measure marketing effectiveness across a multitude of media types, offline- and online channels and campaigns. It has also become more important to have a consolidated view of all marketing and non-marketing activities. MediaMarktSaturn and Neustar set out to answer the question how much incremental value is generated by 120 marketing activities of MediaMarktSaturn, many of which are happening simultaneously.

Methods & Data: The solution uses MMM (Marketing-Mix Modeling) and MTA (Multi-Touch Attribution) statistical models to analyze the data. It integrates a huge variety of data types from sales volume, spends and online tracking data to survey-based funnel KPIs, weather data and store traffic counts. To account for the complexity of the data, a hierarchical Bayesian approach is used.

Results: The effects of marketing activities on brand KPIs, web and store traffic and online and offline sales are being analyzed coherently. It can be shown that different campaign types and media touchpoints influence these KPIs in different ways, opening the opportunity to optimize on specific campaign goals. Survey-based market research results can proxy long-term brand health in the process. The measured efficiency of media types supports a balanced media mix, moving towards the use of online channels, however, it also shows that media types like out-of-home and radio still play a strong part.

Added Value: The study proves that it is possible to measure the incrementality of offline and online marketing activities in a non-experimental environment. It allows MediaMarktSaturn to compare all marketing activities on the basis of ROI and optimize its media budget accordingly. The introduction of survey-based brand kpis provides the basis for steering multiple outcomes. Last but not least the study supplies further evidence of the utility of hierarchical Bayesian methods in tackling imperfect and highly differentiated data.

RELEVANCE & RESEARCH QUESTION

On e-commerce websites such as Amazon, customers readily comment on product highlights and flaws. This provides an important opportunity for companies to collect customer feedback. Fine-grained analyses of customer reviews can support managerial decision-making, especially in marketing. The vast amount of user-generated content necessitates the application of automated analysis techniques. One method of computationally processing unstructured text data is sentiment analysis, which examines people’s opinions, evaluations, emotions, and attitudes towards products, services, organizations, or other topics (Liu 2012, p. 1). Past studies have primarily focused on document-level or sentence-level sentiment analysis. However, for practical applications, there is a substantial need for finer-grained analyses to determine what exactly customers like or dislike – thus, for aspect-based sentiment analysis (ABSA). However, the implementation of ABSA is challenging.

The need for ABSA in marketing, the limitations of traditional sentiment analysis methods, and recent progress in the field of artificial neural networks make the latter’s application to ABSA a relevant research topic.

The objectives of this thesis are to

̶ propose and evaluate a novel model architecture for ABSA that combines gated recurrent units and convolutional neural networks;

̶ apply the proposed model to laptop reviews to gain insight into customer requirements and satisfaction;

̶ discuss limitations of the proposed model, also from a market research perspective.

'

'

METHODS & DATA

ABSA is divided into the subtasks of aspect term extraction and aspect sentiment classification. One neural network was trained to predict for each word of every sentence whether the word is an aspect. All aspects and their corresponding sentences were passed on to a second neural network, which predicts whether the sentiment expressed regarding the aspect is positive, negative, or neutral. Building on previous research, this thesis suggests combining two artificial neural network types, namely gated recurrent units and convolutional neural networks.

The proposed model was trained using the SemEval 2014 laptop review dataset (SemEval 2014). In addition, the thesis author manually annotated laptop reviews. The combined dataset consists of 5,165 sentences, totaling approximately 72,900 words. To evaluate model performance compared to previous research, the proposed model was trained only on the original SemEval training set and tested on the SemEval test set.

'

'

RESULTS

The evaluation results (Fig.1, 2) are promising. Without using sentiment lexica, handcrafted rules or manual feature engineering, the proposed system achieves competitive results on the benchmark dataset. It is especially effective at extracting aspects.

Model F1 score

Proposed model 81.63%

Filho and Pardo (2014) 25.19%

Pontiki et al. (2014) 35.64%

Toh and Wang (2014) 70.41%

Chernyshevich (2014) 74.55% †

Liu, Joty, and Meng (2015) 74.56%

Poria, Cambria, and Gelbukh (2016) 77.32%

Xu et al. (2018) 77.67%

Fig. 1: Performance on aspect term extraction on the SemEval test set

†: trained on twice as much training data, use of an additional training set

'

To ensure comparability, only the performance of models with publicly available word embeddings are reported in Fig. 1. With domain-specific word embeddings and a set of linguistic patterns, Poria, Cambria, and Gelbukh (2016) reached an F1 score of 82.32%, which appears to be the current state of the art for this task.

'

Model | Accuracy | Macro F1 score

Proposed model | 68.45% | 63.92%

Pontiki et al. (2014) | 51.37% | n/a

Negi and Buitelaar (2014) | 57.03% | n/a

Wang et al. (2016) | 68.90% | n/a

Wagner et al. (2014) | 70.48% | n/a

Kiritchenko et al. (2014) | 70.48% | n/a

Tang et al. (2016) | 71.83% | 68.43%

Chen et al. (2017) | 74.49% | 71.35%

Fig. 2: Performance on aspect sentiment classification on the SemEval test set

'

Sentiment misclassifications can be grouped into three types of mistakes: predicting the opposite sentiment, predicting a strong sentiment instead of neutrality, and predicting neutrality instead of a strong sentiment. For marketers who interpret the model predictions, the first type of mistake would be the most severe. The third type of mistake was most common. It is arguably the least severe mistake and shows that the proposed system tends to be conservative in its predictions.

Overall, model performance is encouraging, especially because the model used only two features. This is in sharp contrast to traditional methods. Moreover, no specialized knowledge of linguistics was needed to develop the proposed system. In addition, it does not use any sentiment lexica, which is especially beneficial when considering languages other than English.

A case study in the laptop domain illustrates how and to what degree the proposed ABSA system is useful for practical purposes in market research.

'

'

ADDED VALUE

The paper’s contributions are

̶ provision of a labeled dataset for ABSA, which could enhance other models;

̶ provision of refined annotation guidelines that consider marketing needs;

̶ proposal and implementation of a system that combine gated recurrent units and convolutional neural networks;

̶ performance evaluation of the system;

̶ error analyses, which can help practitioners to interpret the model output and may allow academics to improve future models;

̶ model outputs that summarize the customer opinions voiced in unlabeled and unstructured reviews;

̶ some insight into customer satisfaction and preferences (regarding the case study laptops), which might facilitate decision-making in marketing;

̶ guidance on why, how, and under what limitations to use ABSA, especially for marketing purposes.

With only words and part-of-speech tags as inputs, the proposed system achieves competitive results on the benchmark dataset. Sentiment lexica, handcrafted rules or manual feature engineering are not required. The system can be readily used to analyze English customer reviews of laptops. Given appropriate training data, the approach may also be applicable to other product categories and languages.

ABSA offers a structured representation of the most frequently mentioned positive and negative aspects in customer reviews. Moreover, it does so in a timely manner. The output can help to determine what reviewers like and dislike about a product. Given a large amount of review text, ABSA provides a detailed picture of customer satisfaction and can stimulate product improvements. It can also support marketers in inferring the reviewers’ reasons to purchase the product and the purposes for which they use it. Moreover, ABSA can complement traditional marketing research, especially as a preliminary study or by providing up-to-date information. In short, it can help companies to understand customers.

'

'

REFERENCES

see thesis

RELEVANCE & RESEARCH QUESTION:

Individual data that are collected when using smartphone applications or other digital technologies may not only be used to improve recommendations or products provided to the user. Many of these data may at the same time be employed for a public use, for instance when data collected in navigation apps are used by municipalities for urban planning. Against this background, the advancement of smart technologies opens new possibilities for the provision and maintenance of public goods, such as public health care and infrastructure development. However, such practices entail considerable ethical and social challenges, particularly with respect to privacy violations. From empirical research on privacy norms, we know that individual perceptions of which data transmissions are acceptable heavily depend on situational characteristics and their interactions. Thus, while individuals may accept data uses from which they receive an immediate personal benefit, it is unclear under which conditions using the same data for a public benefit is considered appropriate as well. To investigate this issue, the present paper draws on and advances the application of the theoretical framework of privacy as contextual integrity (Nissenbaum 2004, 2018) that conceptualizes the context dependence of privacy norms. It proposes concrete situational characteristics that impact the forming of these norms: data type, involved actors, and the terms of data transmission. These characteristics interact, meaning that in most cases no single situational feature can explain norms on its own. The question is: Do individuals hold different norms of appropriateness for private and public benefit data uses, and on which concrete situational characteristics does acceptance depend?

METHODS & DATA:

A factorial survey experiment (“vignette study”) was employed in a German online non-probability sample with 1,504 respondents to compare how personal normative beliefs of appropriateness of specific data transmissions are affected by concrete situational characteristics. The vignettes, i.e. situations shown to the respondents, varied along the parameters of data type, data recipient, and use for a private or public benefit. The investigated data types were health, location, and energy use data, while the recipient was either a public administration or a private company. Moreover, general privacy concerns, perceived sensitivity of the three data types, as well as trust in public and private entities were measured to account for possible moderating effects.

RESULTS:

Results of linear regression analyses show that whether respondents perceive a public benefit use of their data as appropriate – compared to a personal benefit use – strongly depends on the concrete data type scenario at hand. In line with the notion of contextual integrity, considerable interactions of situational characteristics are present, i.e. the effects of use vary with data type and recipient. Particularly, public benefit uses are more accepted when public instead of private actors use the data. These findings show that the acceptability of a public benefit use is context dependent and support a contextual conceptualization of privacy norms.

Moreover, normative beliefs can partly be explained by individual characteristics. Interestingly, in two out of three data type scenarios, general privacy concerns decreased acceptance, suggesting that more general privacy sentiments are not always obliterated by concrete situational factors. Perceived sensitivity of a given data type and general privacy concerns strongly contribute to the explanation of variance in normative beliefs, i.e. they appear as major influential factors. However, interactions between a given data type and its sensitivity as well as the interaction between recipient and trust in the recipient were miniscule. Therefore, the data do not allow the inference that higher trust or higher perceived sensitivity strongly altered the impact of recipients or data types.

ADDED VALUE:

The present study contributes to an extension of the application of the contextual integrity framework for privacy norms and offers first insights into conditions under which using data for public benefit uses may be deemed appropriate. It suggests to cautiously design data transmissions of individual data for public benefit uses, particularly as the advancement of possibly invasive technologies promises improvements of the provision of public goods. No one-fits-all preference for public benefit uses of individual data exists but, importantly, public actors are preferred recipients for such a data use. This study paves the way for future investigations of data sharing for the public good and argues to further investigate interindividual differences as drivers of normative beliefs. Furthermore, research and practice will profit from examining additional data sharing scenarios as well as behavioral implications of privacy norms for public benefit use contexts.

REFERENCES:

Nissenbaum, Helen (2004): Privacy as Contextual Integrity, Washington Law Review 79(1): 119–157.

Nissenbaum, Helen (2018): Respecting Context to Protect Privacy. Why Meaning Matters, Science and Engineering Ethics 24(3): 831–852.

Relevance & Research Question:

As ever more surveys are being conducted, respondents recruited for a survey are more likely to already have previous survey experience. Furthermore, it becomes harder to convince individuals to participate in surveys and thus incentives are increasingly used. Both having previous survey experience and participating in surveys due to incentives have been discussed in terms of their links with response quality. In both cases, theoretical arguments exist that argue these factors could increase or decrease response quality. Empirical evidence is scarce and findings are mixed. This study thus aims to shed more light on the link of previous survey experience and participating due to incentives with response quality.

Methods & Data:

We analysed data of the probability-based CROss-National Online Survey (CRONOS) panel covering Estonia, Slovenia and Great Britain. We use three response quality indicators (item nonresponse, occurrence of primacy effects and nondifferentiation) as outcome variables and indicators for having previous web survey experience and being motivated to participate due to the incentive as predictors of main interest in our regression models.

Results:

We found that previous web survey experience had no impact on item nonresponse and occurrence of a primacy effect but reduced nondifferentiation. Being motivated to participate by the incentive did not have a significant impact on any of the three response quality indicators. Hence, overall we find little evidence that response quality is impacted by either of the two factors, previous web survey experience and participating due to the incentive, which are increasingly present in target populations these days.

Added Value:

We add to the scarce pool of empirical evidence on the link between having previous survey experience and participating in surveys due to incentives with response quality. An explicit measure of extrinsic motivation is used in contrast to previous research which often simply assumes extrinsic participation motivation to play a role if incentives are provided.

Relevance & Research Question:

Response rates in surveys have been declining in various disciplines in the last decades. Online surveys have become more popular due to their fast and easy implementation, but they are especially prone to low response rates. At the same time, the increased use of the internet may also lead to a higher acceptance of online surveys in the course of time. Thus, the overall time trend in response rates of psychological online surveys is in question. We hypothesize a decrease in response rates and examine possible moderators of this time effect, as the invitation mode or contact protocols.

Methods & Data:

We searched PsycInfo and PubPsych with the search terms: (Online Survey or Web survey or Internet survey or email survey or electronic survey) and (response rate or nonresponse rate). This resulted in 913 hits. The abstracts of these records were screened to exclude studies not reporting results of online surveys. We excluded 84 records, resulting in 829 articles for full text screening.

So far, 318 full texts have been screened to assess, whether an article reports response rates of online surveys only and if the information on the participant flow is sufficient to compute response rates, the outcome of interest. Using the metafor package in R, mixed effects multilevel models will be used to investigate the hypothesized time effect and the moderating effects of recruitment, invitation and contact protocols.

Results:

With information on 95 samples from 83 reports, the estimated mean response rate in this meta-analysis is 46 %. There is evidence of declining response rates over time. Personal contact to invite respondents improves the willingness to participate. The response rates are slightly higher in samples that received a pre-notification. The number of reminders has no effect on response rates.

Added Value:

The results of the meta-analysis are important to guide decisions on the conduction of online surveys in psychology. To reach the target population, personal contact and use of a pre-notification is recommended. Further investigations, for example on the use of incentives and on the effect of respondent burden, will follow.

Relevance & Research Question:

Attrition has been one of the main targets of survey research in panels from its beginnings (Lazarsfeld 1940, Massey and Tourangeau 2013).

This research aims at using different tools measuring motivation (closed items, paradata, open-ended questions…) to understand how initial motivation, interacting with different types of incentives, plays a part in the decision to leave the panel in the context of changing panel functioning.

Methods & Data:

ELIPSS panel, whose panelists were equipped for survey completion with dedicated tablets connected to the internet, was active from 2012 to 2019. Panelists’ motivations to join the panel were systematically measured during recruitment. Their response quality and motivation were also assessed using paradata and recurrent survey measures. As the panel’s functioning was about to change, an experiment on using unconditional differential incentives to encourage staying in the panel was designed. Three randomly selected groups of panelists were formed. The first received financial incentives of the same amount repeatedly: at t1 and four months later at t2. The second received the same financial incentive once at t2, and the third received the same financial incentive coupled with a gift at t2. We analysed the influence of those incentives on panel attrition in interaction with motivation indicators.

Results:

Initial motivation studies in the ELIPSS panel have outlined the importance of this indicator in further behavior inside the panel. We observed that as panelists declared themselves more interested by incentives, their chances of leaving increased. Results of the experiment show a marginaly significant effect of incentive type on attrition. Panelists who were given additional incentives didn't seem to remain more often, quite the contrary. However, interpretation of those effects should be clearer when taking motivation into account. Certain types of incentives may have differential results on different types of panelists.

Added Value:

The study results add to the knowledge of how the effects of initial motivation to join an online, non commercial research panel interact with unconditional differential incentives to influence panel attrition in an exceptional context. It is thus of primary importance for panel management purposes.

Relevance & Research Question:

The conception and realization of medium- and long-term market research online communities confront researchers with great challenges: In particular qualitative communities are dependent on committed participants staying during the whole field time in order to gain insightful, relevant answers. Some community projects seem to fail precisely in keeping people motivated over a longer period of time.

Through other studies we have already learned that monetary incentive plays an important, but not all determining role. What other factors are relevant? We want to explore the reasons for commitment in research communities and how it can be positively influenced to achieve comprehensive participation throughout a longer community. We focus on the field time, on everything happening after recruitment. We want to better understand how a good online community works from the participants' perspective.

Methods & Data:

We apply a mixed approach and conduct our research in 3 phases with increasing depth: A quantitative questionnaire with about 200 participants is followed by a short digital qualitative project (e.g. forum, associative and projective tasks) with about 40 participants and 8-12 one-hour webcam interviews. This iterative approach allows a gradual identification of participants' experiences, expectations and needs in relation to online communities. We work with partici-pants from different recruitment sources (panel and field service providers), all already took part in qualitative digital research. The total field time is about 2 weeks.

Results:

None yet. We will conduct the study in January-February 2020 and will be able to present the results at the conference.

Added Value:

Our industry is facing the challenge of continuing to attract people who take the time to answer our questions – even over a longer period of time. Especially in qualitative research communities we need open dialogue partners who are willing to stay a while with us. Without participants, there is no market research. Motivated participants who feel valued are likely to provide personal, authentic answers. Understanding why people stay in online communities is important to ensure that those needs are met and people continue their participation.

Relevance & Research Question:

In the last 2 years, substantial improvements in many natural language processing tasks were achieved by building models with several 100s of millions of parameters and pre-training them on massive amounts of publicly available texts in an unsupervised manner [1] [2]. Among the tasks are also many classification problems, which are similar to verbatim coding, the categorization of free-text responses to open-ended questions popular in market research. We wanted to find out if these new models could set new state-of-the-art results in automated verbatim coding and thus potentially be used to accelerate or replace the tedious manual coding process.

Methods & Data:

We train a model based on [1] but adapted to the task of verbatim coding through architecture tweaks and pre-training on millions of reviewed verbatims on https://caplena.com. This model was benchmarked against simpler models [3] and widely available tools [4] as well as against human coders on real survey data. Chosen datasets were provided by two independent market research institutes in the Netherlands (Dutch) and in Brazil (Portuguese and English) with n=525 and n=5900 respectively to test the new model on small surveys and large ones alike.

Results:

Our model was able to outperform both our previous simpler models as well as standard online tools on a variety of surveys in multiple languages with a weighted-F1 improvement on the Brazilian data from 0.37 to 0.59. We achieve new state-of-the art results for automated verbatim coding matching or even surpassing the intercoder agreement between human coders with an F1 of 0.65 from our model vs 0.61 for the human intercoder agreement on the Dutch dataset.

Added Value:

Researchers can now have a tool that performs verbatim coding almost instantaneously and at a fraction of the cost with similar accuracy to full human coding. Besides the quantitative benchmark results, we also provide qualitative examples and guidance as to which surveys are well suited for automated coding and which less so.

References:

[1] https://arxiv.org/abs/1810.04805

[2] https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf

[3] Support-Vector-Machines with Bag-of-Words-Features and Long-Short-Term-Memory Networks with Word-Embeddings

[4] https://monkeylearn.com/

Relevance & Research Question:

Modeling online behavior is a common tool for studying social “ecologies” in digital environments. For this purpose, most of the existing models are used for analysis and prediction.More prediction tools are needed for providing evidence especially for exploring online social behavior to depict the right target group within the right contexts. As a prominent use case In higher education research, the quality of learning processes shall be ensured by providing suitable instruments like mentoring for the pedagogical and social support of students. But how can we identify mentoring needs in digital learning environments? And to what extent can predictive models contribute to the quality assurance of learners’ progress?

Methods & Data:

For contributing to the research questions, we firstly extract and analyze text data from online discussion boards in a distance learning environment by using (automated) text mining procedures (e.g. by topic modelling, semantic analytics, and sentiment analytics). Based on the results, we identify suitable behavioral indicators for modeling specific mentoring occasions by using network analytic instruments. To analyze the emergence of (written) language and to explore latent patterns of sentiments within students’ discussions, GAT (discourse analytic language transcription) instruments are applied. Finally, we build a predictive model of social support in digital learning environments and compare the results with findings from neurolinguistic models.

Results & Added Value:

The contribution proposes an analytical implementation framework for predicting procedures to handle behavioral data of students in order to explore social online behavior in digital learning environments. We present a multidisciplinary model that implements theories from current research, depicts behavioral indicators, and takes systemic properties into account, so that it can be used across the board for applied research. In addition, the findings will provide insights on social behavior online to implement suitable pedagogical and social support in digital learning environments. The study is still a work in progress.

Relevance & Research Question:

Many companies are struggeling with the growing amount of customer data and touchpoint-based feedback. Instead of learning from this feedback and instead of using it to continuously improve their processes and products, many tend to waste it. Besides, traditional causal models like regession analyses do not help to truely understand why KPIs end up at a certail level. Our research objective was to develop a new process, based on AI models, that help to...

a) most automatically structure and analyze customer feedback.

b) better understand customers incl. hidden drivers of their behaviour.

c) help companies to take action based on their customer feedback.

Methods & Data:

Our approach was to make use of information that almost every company has: the NPS score and open-ended text feedback (=reasons for giving this NPS score per customer). The text feedback is being coded by means of supervised learning incl. the sentiment behind statements (=NLP), then the outcomes are being analyzed by means of neural network analysis.

Results:

The explorative nature of this approach (= open-ended feedback and machine learning algorithms) reveals, which influence individual criteria will have on the NPS. Unexpected nonlinearities and interactions can be unveiled. Hidden Drivers to leverage the NPS can be uncovered. A large amount of data is reduced to the most significant aspects, we also developed a simple dashboard to illustrate these.

Added Value:

We found that this approach produces a far better explanation power than traditional methods like manual coding and linear regression - usually, the explanation power is twice as good! Besides, these analyses may be implemented for any industry or product, and they can produce insights for historical data. Finally, when dealing with big data sets, the machine learning approaches help to be faster and more efficient.

Relevance & Research Question:

Analyzing voice content in large scale is a promising but difficult task. Various sources of voice data exist, ranging from audio streams of YouTube videos to voice responses in online surveys. While automated solutions for speech-to-text transcription exist, the question remains how to validate, quality-check and leverage the output of these services to provide insights for market research. This study focuses on methods for evaluating quality, filtering and processing of the texts returned from automated transcript solutions.

Methods & Data:

The feasibility and challenges of processing text transcripts from voice is assessed along the process steps of error recognition, error correction and content processing. As a baseline, a set of 400 voice responses, transcribed with Google’s Cloud Speech-to-Text service, are enriched with the real responses taken from original audio. Differences, being errors in recognition, are categorized into their different causes, such as unknown names or similar sounding words. Different methods for error detection and correction are proposed, applied and tested against this goldset corpus and evaluated. The resulting texts are processed with an NLP concept detection method usually applied to UGC content in order to check how further insights such as inherent topics can be derived.

Results:

Although having improved substantially over the past years, handling speech-to-text output remains challenging. Unsystematic noise generated from mumbling or individual pronunciation is less problematic, but systematic errors such as mis- or undetected person or brand names, due to difficult or ambiguous pronounciation, may distort results substantially. The error detection methods shown, along with detection confidence values retrieved from the transcription service, provide a first baseline for filtering, rejecting input of low quality, and further processing in order to get meaningful insights.

Added Value:

Previous studies have shown that voice responses are more intense and longer than typed ones. Having ways of evaluating and controlling the output of speech-to-text services, knowing their limitations, and checking the applicability of NLP methods for further processing is vital to build robust analytical services. This study covers the topics that have to be addressed in order to draw substantial benefits from voice as a source of (semi-)automated analytics.

Relevance and research question:

Today, social media like Facebook and Whatsapp allow anyone to produce and disseminate “news”, which makes it harder for people to decide which sources to trust. While much recent research has focussed on the items of (mis)information that people believe, less is known what makes people trust a given. We focus on three source characteristics: Whether the source is known (vs. unknown), on what channel people receive its content (Facebook vs. web site) and whether previous information by that source was congruent (vs. incongruent) with someone's worldview.

Methods & Data:

In an pre-registered online survey experiment with a German quota sample (n = 1980), we expose subjects to a series of news reports, manipulated in a 2x2x2 design. Through the use of HTML, we create highly realistic stimulus material that is responsive to mobile and desktop respondents. We measure whether people believe that a report is true, and whether they would share it.

Results:

We find that individuals have a higher level of belief in and a somewhat higher propensity to share news reports by sources they know. Against our expectation, the source effect on belief is larger on Facebook than on websites. Our most crucial finding concerns the impact of congruence between facts and subjects’ world view. People are more likely to believe a news report by a source that has previously given them congruent information—if the source is unknown.

Added value:

We re-evaluate older insights from the source credibility literature in a digital context, accounting for the fact that social media has changed the way sources appear to news consumers. We further provide causal evidence that explains why people tend to trust ideologically aligned sources.

Relevance & Research Question:

It seems nostalgic to recall that the early days of the internet inspired hopes for a more egalitarian and democratic society. Some of this promise has been fulfilled — connectivity has enabled new forms of collective mobilization and made human knowledge accessible to anyone. But we are also living with the side effects of the Internet. Among them, pervasive disinformation in the polity that is weakening the integrity of our democracies and bringing people to the streets.

Fake news, hostile disinformation campaigns and polarization of the political debate have combined to undermine the shared narrative that once bound societies together. Trust in the institutions of democracy has been eroded. Tribalism and a virulent form of populism are the hallmarks of contemporary politics. The rules of politics are being rewritten.

Conducted as part of multi-stakeholder dialogue with the social media platforms on the renovation of democracy, our research explores the impact of social media on democratic society AND the impact of democratic disruptions on the reputation of social media platforms themselves.

Methods

20 minute surveys conducted in June and July 2019 in France, Germany and UK (n=500 in each country, representative age and gender). All agreed to install a software that monitors their online activity. They all have been tracked for 12 months before they participated in the research

Kmean segmentation combining declarative and passive data. Declarative data includes the attitude of each respondent towards “traditional” fake news. Passive data is mainly focused on the types of sites where they find information (main news websites or user generated content for instance).

Results

This research reveals three paradoxes of the digital democracy.

1. Those who supported, and who benefited from the digital revolution the most are those who trust the GAFAs the least, and think they now need to be controlled.

2. Those who trust democratic institutions the least are those who believe in Facebook political virtues the most.

3. To believe in fake news is less a cognitive matter than a political statement.

Added value

This research describes mechanisms by which facebook takes advantage of fake news.

Relevance & Research Question:

In the age of social media, the questions of what is allowed to say and how hate speech should be regulated are ever more contested. We hypothesize that content- and context-specific factors influence citizens’ perceptions of the offensiveness of online content, and also shape preferences for action that should be taken. This has implications for the legitimacy of hate speech regulation.

Methods & Data:

We present a pre-registered study to analyze citizens’ preferences for online hate speech regulation. The study is embedded in nationally representative online panels in the US and Germany (about 1,300 respondents, opt-in panels operated by YouGov). We construct vignettes in forms of social media posts that vary along key dimensions of hate speech regulation, such as sender/target characteristics (e.g., gender and ethnicity), message content, and target’s reaction (e.g., counter-aggression or blocking/reporting). Respondents are asked to judge the posts with regards to their offensiveness and consequences the sender should face. Furthermore, the vignette task was embedded in a framing experiment, motivating it by (a) looming government regulation protecting potential victims of hate speech, (b) civil rights groups advocating against censorship online, or (c) a neutral frame.

Results:

While for half (48%) of the posts the respondents saw no need for action by the platform provider, for 11% of the posts they would have liked to see the sender to be banned permanently from the platform. Violent messages are substantively more critically evaluated than insulting or vilifying messages. At the individual level, we find that females are significantly more likely to regard the posts as offensive or hateful than males. With regards to the framing experiment, we find that, compared to the control group, respondents confronted with the government prime are 20pp less likely to demand no action in response to offensive posts.

Added Value:

While governments around the world are acting towards regulating hate speech, little is known about what is deemed acceptable or inacceptable speech online in different parts of the population and societal contexts. We provide first evidence that could inform future debates on hate speech regulation.

Relevance & Research Question:

Advertising impact research for a spot of a new OTC product played on digital displays at various touchpoints (DOOH). The main focus of the study was to investigate the increase of KPIs achieved with DOOH for several target groups as well as to determine the contribution of individual touchpoints. Who sees advertisements, when and how receptive is a person to this TP is a central question.

Methods & Data:

A combination of methods was used: online survey, geo location tracking and a follow-up survey. The samples were drawn according to representative structural data. The follow-up survey comprised n = 2 x 2,000 participants. One part (n=2000/mobility panel) was asked to track their movement for 21 days throughout the campaign period. Additionally, for this sub-sample we had information about its values and settings. After the campaign, both samples were interviewed in the online follow-up survey on various parameters of product and advertising awareness.

Results:

For the mobility panel, the combination of tracking and survey data enabled the recall of the advertisement to be determined with additional information on the actual advertising contact opportunities and frequencies. Additionally, the question about target group specifics of individual touchpoints could be answered. The comparison with participants who were not tracked shows the high value of the mobility data. It allows location and time dependent programmatic modulation of the advertising. The content of each touchpoint therefore can be adjusted specifically for certain target groups and its effect can be analysed to e.g. reduce scattering losses, which ultimately can increase its recall and recognition and thus the ROI.

Added Value:

Geo-tracking enables the continuous mapping of the movement patterns of target groups. In combination with the described profile data, incl. decision-making style, it allows precise knowledge about different target groups. This leads to a significantly improved accuracy in addressing target groups and advertising effectiveness of touchpoints. It will also enable further approaches to understanding target groups and their behaviors, as it is possible to identify people who came in contact with not only specific touchpoints but also POIs, making it possible to determine the impact of contact with a brand on brand relevance and image.

Relevance & Research Question: The substantial increase from 8 to more than 90 products in around 40 years, increased the complexity of the product production and preparation process of McDonald’s Switzerland, and consequently threatens the customer-perceived product quality. The critical research challenge and insights goal was to regain efficient resources through removing selected products from the menu but without threatening customer growth.

Methods & Data: In order to ensure valid and reliable insights with a precise predictive forecasting power, a set of highly advanced methods was applied. In detail, several advanced implicit association tests regarding a brand and product assessment has been employed online and an online-based discrete choice experiment that simulated various consumption situations depending on the shown task-related behavior before with up to 5 choice tasks. In total, a set of five strongly behavior-related indicators (e.g., implicit brand-product-fit, purchase potential etc.) has been derived. Finally, based on that indicators, a final single value indicator for each investigated product has been calculated to determine the overall customer-related consumption value that has been further utilized to derive a review list of products to identify the products that have no substantial impact on the customer’s future behavior. In total, 2056 customers of McDonald’s aged 15 to 79 years who visit a McDonald’s restaurant in the last 12 months at least one time have been interviewed.

Results: Based on the derived product review list, selective products in the bottom have been removed from the market. Before, a swiss-wide field experiment in more than 30 restaurants has been conducted to check the actual behavior. As predicted, no decrease related to customer’s visit frequency and average menu spending could be identified. Subsequently, the respective products have been removed from all restaurants.

Added Value: The innovative method combination was capable to identify customer’s product value, and thus customer’s behavior in the real world. Before, other country organizations of McDonald’s had tried before just one single method (e.g., menu choice based conjoint), but failed. Currently, an increasing number of other country organizations is applying this developed approach for a strengthened market performance.

Social media platforms (SMPs) influence the communication of virtually all stakeholders in democratic politics. Politicians and parties campaign through SMPs, media organizations use them to distribute political news, and many citizens read, share, and debate political issues across multiple social media accounts. When assessing the political implications of these practices, scholars have traditionally focused on the commonalities of SMPs, rather than their differences. The implications of this oversight are both theoretical and methodological. Theoretically, scholars lack an overarching conceptual framework to inform cross-platform research designs. As a result, the operationalization of social media variables across platforms is often inconsistent and incomparable, limiting the attribution of platform-specific effects.

This dissertation therefore contributes to the study of digital media and politics by developing a cross-platform theory of platforms’ digital architectures. Digital architectures are defined as the collective suite of technical protocols that enable, constrain, and shape user behavior in a virtual space. The dissertation’s central argument is that the digital architectures of SMPs mediate how users enact political processes through them. Focusing on politicians’ campaigning and citizens’ political participation during elections, I show how political communication processes manifest differently across platforms in ways that can be attributed to their digital architectures. Moreover, I demonstrate how both politicians and citizens manipulate the digital architectures of platforms to further their political agendas during elections.

To mount these arguments, the dissertation adopts a highly conceptual, exploratory, and interdisciplinary approach. Its main theoretical contribution, the digital architectures framework, brings together fragments from literatures spanning archeology, design theory, media studies, political communication, political science, social movements, and software engineering. Methodologically, the study combines qualitative and quantitative methods to address the research questions of four individual research articles (Chapters 4-7). These studies have been published in Information, Communication & Society; Journalism & Mass Communication Quarterly; Language and Politics; and a book chapter in Social Media and European Politics (Palgrave). The main empirical cases included in the dissertation are the 2015 British General Election, the 2016 Brexit Referendum, and the 2016 U.S Presidential Election.

The structure of the dissertation is as follows. Chapter 1 introduces the dissertation’s overarching research questions and design. Chapter 2 situates the digital architectures framework within the existing literature by critiquing existing theoretical approaches to social media and political participation. Chapter 3 outlines the main challenges in studying participation on social media, as well as summarizes the dissertation’s methodological approach. Chapter 4 then presents the digital architectures framework through a systematic, cross-platform comparison of Facebook, Twitter, Instagram, and Snapchat. In this chapter, I illustrate how the digital architectures of these SMPs shaped how American politicians used them for political campaigning in the 2016 U.S. election.

Shifting focus from politics to citizens, Chapter 5 examines how the digital architectures of social media influence citizens’ political participation. Chapter 5 characterizes the various styles and degrees of political participation through SMPs, and it shows how the architectures of Twitter and Facebook lead to different manifestations of online participation in the context of European politics.

Building on the Chapter 5’s conceptual work, Chapters 6 and 7 use digital trace data to empirically investigate citizens’ participation on Twitter and Facebook, respectively. Chapter 6 offers a new theory of online political participation by conceptualizing it as a process, rather than as an activity. Chapter 6 develops a typology of political participation and applies it to citizens’ use of Twitter in the 2015 British General Election. We find that a small number of highly active citizens dominate the political discussion on Twitter, and these citizens tend to promote right-wing, nationalist positions.

Chapter 7 finds similar patterns in citizens’ political participation on Facebook during the 2016 Brexit referendum. Using metadata to chart the commenting patterns of citizens across media and political Facebook pages, Chapter 7 reveals that Leave supporters were much more active in political commentary than Remain supporters. However, this phenomenon is, again, due to a small number of active citizens promoting right-wing, nationalist positions. Few citizens commented on both media and campaign Facebook pages during the referendum, but those who did commented on the media first. This finding, together with the observation that political commentary overwhelmingly took place on media pages, supports the notion that the mainstream media maintain their agenda-setting role on SMPs.

Lastly, Chapter 8 argues that different digital architectures afford varying degrees of publicness, which in turn affects how political participation is actualized across platforms. This chapter, and the thesis, concludes with a discussion of why the digital architectures of social media are critical to consider when assessing social media’s impact on democracy.

The present PhD thesis lies within the area of Internet-based research; specifically it is concerned with Internet-based assessment via questionnaires and smartphones. The thesis investigates the influence of response scales and objective sensor data from smartphones on the data gathering process and data quality. Internet-based questionnaires and tests are becoming increasingly common in psychology and other social sciences (Krantz & Reips, 2017; Wolfe, 2017). It is therefore important, for researchers and practitioners alike, to base decisions about their research design and data gathering process, on solid empirical advice.

The first research article compared two types of response scales, visual analogue scales (VASs) and Likert-type scales, with regard to non-response. Participants of an eHealth intervention were randomly allocated to answer an extensive questionnaire with either VASs or Likert-type scales as response options to otherwise identical items. A sample of 446 participants with a mean age of 52.4 years (SD=12.1) took part. Results showed lower SDs for items answered via VAS. They also indicated a positive effect of VASs with regard to lowering dropout of participants, OR=.75, p=.04.

The second research article investigated the validity of measurement, again comparing VASs and Likert-type scales. The response scales of three personality scales were varied in a within-design. A sample of 879 participants filled in the Internet-based questionnaire, answering the personality scales twice in a counterbalanced design. Results of Bayesian hierarchical regressions largely indicated measurement equivalence between the two response scale versions, with some evidence for better measurement quality with VASs for one of the personality scale Excitement Seeking, B10=1318.95, ΔR2=.025.

The third research article investigated the validity of objective sensor data in an experience sampling design. The association of subjective well-being with smartphone tilt was investigated in two separate samples implementing different software to gather data. In both samples measurements consisted of cross-sectional questionnaires and a longitudinal period of three weeks, with measurements twice per day. Results provided evidence for the validity of smartphone tilt as an indicator of subjective well-being, t(3392)=-3.9, p<.001. In addition to the analysis of tilt and well-being, potential biases and problems when implementing objective data are discussed, specifically when different software implementations and operating systems are involved.

In conclusion, the PhD thesis offers valuable insights on Internet-based assessment. The VAS’s position as a superior response scale was strengthened.Its advantages over more traditional Likert-type scales, e.g., offering better distributional properties and more valid information, were confirmed and no disadvantages emerged. Smartphone sensor data were shown to provide a way to validate self-report measurement, if potentially important caveats related to differences in data are identified and addressed.

Relevance & Research Question:

A problem observed by Lakens, Hilgard, & Staaks (2016) is, that many published meta-analyses remain static and are not reproducible. The reproducibility of meta-analyses is crucial for several reasons. First, to enable the research community to update meta-analyses in case of new evidence. Second, to give other researchers the opportunity to use subsets of meta-analytic data. Third, to enable the application of new statistical procedures and test the effects these have on the results of a meta-analysis.

We plan to set up an infrastructure for the dynamic curation and analysis of meta-analyses in psychology. A CAMA (Community Augmented Meta-Analysis) serves as an open repository for meta-analytic data, provides basic analysis tools, makes meta-analytic data accessible and can be used and augmented by the scientific community as a dynamic resource (Tsuji, Bergmann, & Cristia, 2014).

Methods & Data:

We created templates to standardize the data structure and variable naming in meta-analyses. This is crucial for the planned CAMA to enable interoperability of data and analysis scripts. Using these templates, we standardized data of meta-analyses from two different psychological domains (cognitive development and survey methodology) and replicated basic meta-analytic outputs with the standardized data sets and analysis scripts.

Results:

We succeeded in standardizing various meta-analyses in a common format using our templates and in replicating the results of these meta-analyses with the standardized data sets. We tested analysis scripts for various meta-analytic outputs for the CAMA planned, as funnel plots, forest plots, power plots, and meta-regression.

Added Value:

Interoperability and standardization are important requirements for an efficient use of open data in general (Braunschweig, Eberius, Thiele, & Lehner, 2012). The templates and analysis scripts presented moreover serve as the basis for the development of PsychOpen CAMA, a tool for the research community to collect data and conduct meta-analyses in psychology collaboratively.

Relevance & Research Question:

Various empirical evidence signals that general attitudes towards surveys do predict willingness to participate in (online) surveys (de Leeuw et al. 2017; Jungermann/Stocké 2017; Stocké 2006). The nine-item short form of the Survey Attitude Scale (SAS) as proposed by de Leeuw et al. (2010, 2019) differentiates between three dimensions: (i) survey enjoyment, (ii) survey value, and (iii) survey burden. Previous analyses in different datasets have shown that especially the two dimensions, survey value and survey burden, do not perform satisfactory with respect to internal consistency and factor loadings in different samples (Fiedler et al. 2019). Referring to de Leeuw et al. (2019), we therefore investigate into the question whether the SAS can be further improved by reformulating single items and adding new ones from existing literature (Stocké 2014; Rogelberg et al. 2001; Stocké/Langfeldt 2003).

Methods & Data:

Consequently, we implemented the proposed German version of the SAS, adopted from the GESIS Online Panel (Struminskaya et al. 2015) in an online survey for German Higher Education Graduates being conducted recently (October - December 2019, n = 1,378). Furthermore, we realised a survey experiment with split-half design aiming to improve the SAS by varying the wording of four items and adding one supplemental item per dimension. To compare both scales, we use confirmatory factor analysis (CFA) and measures for internal consistency within both groups.

Results:

Comparing CFA results, our empirical findings indicate that the latent structure of the SAS is reproducible in the experimental as well as in the control group. Factor loadings as well as reliability scores support the theoretical structure adequately. But, we do find evidence that changes in the wording of the items (with respect to harmonize the use of terms and to avoid survey mode mentioning) can partially improve the internal validity of the scale.

Added Value:

Overall, the standardized short SAS is a promising instrument for survey researchers. By intensively validating the proposed instrument in an experimental setting, we contribute to the existing literature. Since de Leeuw et al. (2019) also do report shortcomings of the scale; we show possibilities for further improvement.

Relevance & Research Question: ---Quality criteria in the field of (online) surveys are already existing for quite a long time and are therefore also viewed as well established. With the upcoming of new methodological approaches like new sampling procedures, it's questionable though if those criteria are still up-to-date and how current research achieves a methodological suitable reconstruction of quality. Therefore this contribution deals with the current state of the art in meeting quality criteria of online surveys in times of big data, self-learning algorithms and AI.---

Methods & Data: ---On the basis of a narrative literature review, the current state of research will be presented. Findings from both academic as well as applied research are brought together and transfered into recommendations for action. Current (academic) contributions were analysed for challenges and potentials related to quality assurance procedures in the area of online research. In addition, scientific standards and current codes of conduct for the industry were elaborated and examined for their adaptability and scalability for market, opinion and social research.---

Results: ---The results are currently being processed. However, there still are general quality criteria such as objectivity, reliability and validity. The construct „representativeness“ though is still in discussion and it's not clear yet whether the gold standard of a representative survey does work online without manipulation procedures. Therefore, a new view on quality criteria in online contexts seems essential in order to give orientation for the successfully implementation of new methods of online research in the future.---

Added Value: ---The aim is to make a sustainable contribution in the field of quality assessment for both academic and applied online research, especially online surveys. A particular benefit for applied research is to address problems such as survey fatigue and acceptance issues.---

Relevance & Research Question:

According to the GRIT-report 2019, market-researchers consider research and analysis automation a crucial opportunity for their industry. Although qualitative studies are harder to automate than quantitative ones, the automation of qualitative content analysis, a major qualitative evaluation method, is already partially feasible and expected to be further developed.

Main research question: How can qualitative content analysis be (semi-)automated?

Sub-questions: What would an automated qualitative research process as a whole look like? How does it advance online research?

Methods & Data: Semi-automation of qualitative content analysis as well as the research process as a whole are conceptualized based on the hermeneutic method, which is applied to a non-automated study carried out by the author using case-study methodology. Automation approaches already in use are identified through a systematic literature-review.

Results: Currently, qualitative content analysis and the qualitative research process as a whole can only be semi-automated since they depend on continuous human-machine interaction. Full automation seems feasible with the advance of artificial intelligence. It would be based on online and mobile technologies.

Added Value: This poster highlights a roadmap for the automation of qualitative research comprising qualitative content analysis, an increasingly important topic in qualitative social research, and especially in market research.

Relevance & Research Question: Socially desirable responding (SDR), understood as the tendency of respondents to present themselves in surveys in the best possible light, is often understood as a one- or two-dimensional construct. The two short scales, Egoistic (E-SDR) and Moralistic Socially Desirable Responding (M-SDR) understand SDR as a four-dimensional construct. This understanding represents the most comprehensive conceptualization of SDR. Nevertheless, these short scales have not yet been applied and validated in a general population study. Such an application is important to measure and control for social desirability bias in general population surveys. Therefore, we test the reliability and validity of both short scales empirically to provide a practical measure of the four dimensions of SDR in self-administered surveys.

Methods & Data: The items of the source versions of the E-SDR and M-SDR were translated into German using the team approach. To avoid measuring a response behavior rather than social desirability bias, we balanced negative and positive formulated items. The scales together comprise 20 items. We integrated these 20 items into a questionnaire within a mixed-mode mail- and web-based survey conducted in the city of Mannheim, Germany (N~1000 participants). The sample was selected via Simple Random Sampling (SRS).

We assess the reliability and validity of E-SDR and M-SDR by using different analytical methods. To test the reliability, we aim to compute Cronbach’s alpha, the test-retest stability for the two short-scales, and the item-total correlation. To investigate the validity, we will test the construct validity by confirmatory factor analysis (CFA). For measuring discriminant and convergence validity, we correlate the two short-scales with the Big Five traits Extraversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness.

Results: The field period will be from November 2019 to December 2019, and the first results will be available in February 2020.

Added Value: Based on our findings, we can evaluate the four-dimensional measurement of SDR with E-SDR and M-SDR short-scales in self-administered population surveys. If the measurement turns out to be reliable and valid, it can be used in future general population surveys to control for SDR.

Relevance & Research Question:

A number of studies have shown that probability-based surveys lead to more accurate univariate estimates than nonprobability surveys. However, some researchers claim that, while they do not produce accurate univariate estimates, nonprobability surveys are “fit for purpose” regarding bivariate and multivariate analyses. In this study, we therefore assess to what extent bivariate and multivariate survey estimates from probability-based and nonprobability online panels lead to accurate conclusions.

Methods & Data:

We answer our research question using data from a large-scale comparison study in which three waves of data collection were commissioned in parallel to two academic probability-based online panels and eight commercial nonprobability online panels in Germany. For each of the online panels, we calculate bivariate associations and multivariate models and compare the results to gold-standard benchmarks, examining whether the strength, direction, and statistical significance of the coefficients accurately reflects the expected outcomes.

Results:

Regarding key substantive political scientific variables (voter turnout and voting for the main German conservative party (CDU)), we find that the probability-based online panels in our study generally lead to more accurate associations than the nonprobability online panels. Unlike the probability-based online panels, the nonprobability online panels produce a number of significant associations that are contrary to expected outcomes (e.g., that older people are significantly less likely to vote for the main conservative party). Furthermore, while the two probability-based online panels in our study produce similar findings, there is a lot of variability in the results from the nonprobability online panels and none of them consistently outperform the others.

Added Value:

While a number of studies have assessed the accuracy of univariate estimates in probability-based and nonprobability online panels, our study is one of the few that examine bivariate associations and multivariate models. Our preliminary results do not support the claim that nonprobability surveys are fit for the purpose of bivariate and multivariate analyses.

Relevance & Research Question:

In the era of social media, the huge availability of digital data allows to develop several types of research in a wide range of fields. Such data is characterized by several advantages: reduced collection costs, short retrieval times and production of almost real-time outputs. At the same time, this data is unstructured and unclassified in terms of content. This study aims to develop an efficient way to filter and analyze tweets by means of sentiment related to a specific topic.

Methods & Data:

We developed a semiautomatic unsupervised dictionary-based method to filter tweets related to a specific topic (environment, in our study). Starting from the tweets sent by a selection of Official Social Accounts linked with this topic, a list of keywords, bigrams and trigrams is identified in order to set up a topic-oriented dictionary. We test the performance of our method by applying the dictionary to more than 54 million tweets posted in Great Britain between January and May 2019. Since the analyzed tweets are geolocalized due to the method of data collection, we also analyze the spatial variability of the sentiment for this topic across the country sub-areas.

Results:

All the performance indexes considered denote that our semiautomatic dictionary-based approach is able to filter tweets linked to the topic of interest. Despite the short time window considered, we highlight a growing inclination to environment in any area of Great Britain. Nevertheless, the spatial analysis found a lack of spatial correlation (probably because environment is a broad argument, but also strongly affected by local factors).

Added Value:

Our method is able to build (and to periodically update) a dictionary useful to select tweets about a specific topic. Starting from this, we classify selected tweets and we apply a spatial sentiment analysis. Focusing on environment, our method of setting up a dictionary and of selecting tweets by topic leaded to interesting results. Thus, it could be reused in the future as a starting point for a wide variety of analysis, also on other topics and for other social phenomena.

Relevance & Research Question:

The measurement quality of survey questions, defined as the product of validity and reliability, indicates how well a latent concept is measured by a question. Measurement quality also needs to be estimated in order to correct for measurement error. Multitrait-Multimethod (MTMM) experiments allow us to do this. Our research aims to determine the measurement quality resulting from variations in formal characteristics such as number of scale points and partial versus full labelling of scale points, for the given questions in web mode.

Methods & Data:

We conducted two MTMM experiments on the mixed-mode (majority web) GESIS panel, one dealing with environmental attitudes and the other with supernatural beliefs. We estimate the quality of three different response scales for each of the experiments by means of structural equation modelling.

Results:

We do not have results yet. Based on evidence from face-to-face surveys, we would expect that, in both cases, a continuous scale with fixed reference points will lead to the highest measurement quality among the three, that a partially labelled 11-point scale will result in the second highest measurement quality and that a fully labelled 7-point scale will yield the lowest measurement quality.

Added Value:

Quite some research exists on MTMM experiments in more traditional modes, especially face-to-face. However, only few MTMM experiments in web mode have been conducted and analyzed so far.

Relevance & Research Question:

Online experiments have become a popular way of collection data in social sciences. However, high technical hurdles in setting up a server prevent a researcher from starting an online study. On the other hand, proprietary software restricts the researcher’s freedom to customize or share the code. We present Open Lab – the server-side application that makes online data collection simple and flexible. Open Lab is not dedicated to one particular study, but is a hub where online studies can be easily carried out.

Methods & Data:

Available online at https://open-lab.online, the application offers a fast, secure and transparent way to deploy a study. It takes care of uploading experiment scripts, changing test parameters, managing the participants’ database and aggregating the study results. Open Lab is integrated with the lab.js experiment builder (https://lab.js.org/), which enables the creation of new studies from scratch or the use of templates. The lab.js study can be directly uploaded to Open Lab and is ready to run. Integration with the Open Science Framework allows researchers to automatically store the collected data in an OSF project.

Results:

At the conference, we will present the main features of the web application together with results of empirical studies conducted with Open Lab.

Added Value:

Open Lab enables interdisciplinary projects where behavioral scientists work together, and participants not only play a role of passive subjects, but also learn about the science, talk to a researcher or even propose and implement new versions of the task.

Relevance & Research Question: Survey data collection costs have risen to a point where many survey researchers and polling companies are abandoning large, expensive probability-based samples in favour of less expensive nonprobability samples. The empirical literature suggests this strategy may be suboptimal for multiple reasons, amongst them probability samples tend to outperform nonprobability samples on accuracy when assessed against population benchmarks. However, nonprobability samples are often preferred due to convenience and cost effectiveness.

Methods & Data: Instead of forgoing probability sampling entirely, we propose a method of combining both probability and nonprobability samples in a way that exploits their strengths to overcome their weaknesses within a Bayesian inferential framework. By using simulated data, we evaluate supplementing inferences based on small probability samples with prior distributions derived from nonprobability data. The method is also illustrated with actual probability and nonprobability survey data

Results: We demonstrate that informative priors based on nonprobability data can lead to reductions in variances and mean-squared errors for linear model coefficients.

Added Value: A summary of these findings, their implications for survey practice, and possible research extensions will be provided in conclusion.

Relevance & Research Question: Trust has been an important topic in technology research and adoption of web-based services. Users’ trust is not only essential for acceptance of new systems but also for a continued adoption. As a first step of a systematic approach to design trusted information systems and enable trustful interaction between users and information systems, we develop a new comprehensive model of trust in information systems based on existing literature. This models includes both precursors (e.g., perceived trustworthiness) and consequences of trust in IS (e.g., actual use, forgetting of information stored in the IS for cognitive relief).

Methods & Data: Based on extant literature, we differentiate experienced trust in IS from perceived trustworthiness of an IS, intentions to use an IS, and actual usage of an IS. Moreover, exiting experiences with an IS as well as perceived risk and more general contextual factors are considered as moderating factors. Finally, the resulting model is reflected and refined in light of empirical findings on precursors and outcomes of trust in IS.

Results: Our new comprehensive model not only structures the existing research, but also provides a number of concrete propositions. Perceived IS trustworthiness should affect trust particularly when users already have experience with the IS. Experienced trust in IS should strengthen users’ intention to adopt an IS, but this effect should be counteracted by perceived contextual risks. Trust should be influenced by the users’ personality, e.g., their general disposition to trust. The translation of users’ intentions to use an IS into actual behavior should be affected by enabling (e.g., control) and inhibiting (e.g., existing work routines) contextual factors. Finally, the model contains feedback loops describing how actual use of an IS influences perceived trustworthiness of an IS, and following trust experience of users. Evidence from two experimental studies supports the validity of our model.

Added Value: We provide a new theoretical model that outlines a structured process chain of trust in IS. In addition to the theoretical contribution, this systematic approach provides various practical implications for the design of IS as well as for the implementation and maintenance process.

Relevance & Research Question: Many companies see the lack of skilled workers as a central obstacle to the digital transformation. It is well-known that diverse workforces lead to more balanced decisions and more innovation. Nevertheless, women, for example, are still underrepresented in STEM-professions. The following research question arises: Are there any differences in the perception of relevant competencies for the digitalized working world according to gender, age, employment status and migration background?

Methods & Data: Following preliminary literature research and qualitative expert interviews [n=6], a quantitative study was conducted from Nov. – Dec. 2018. Participants [n=515] were recruited among students and companies using faculty email lists, paper form and social media. Participants were asked to assess a total of 14 competencies, knowledge resources and behaviours in their relevance for the digitalized working world on a 6-tier scale. Correlations were determined by calculating Chi-square according to Person and Cramer’s V. Means were compared using T-Test and Levene.

Results: Respondents consider openness to change (5.50), IT and media skills (5.40) and learning ability (5.36) to be the most relevant. Analytical skills (4.79) and empirical knowledge (4.56) are less in demand.

Men rate innovation competence (χ²=10.895, p=.028, V=0.146), decision-making ability (χ²=13.801, p=.017, V=0.164) and ability to think in context (χ²=14.228, p=.014, V=0.167) slightly higher than women. No correlation can be found regarding respondents’ migration background. Among company representatives, eight competencies are rated significantly higher than by students, especially communicative competence (+0.91) and interdisciplinary thinking and acting (+0.74). Moreover, it is noticeable that older participants (generation X, born 1964-1979) consider all competencies to be more important than younger ones (generation Z, 1996-2009), apart from IT and media competence. The items openness to change (T-Test p=.004, Levene p=.004), self-organisation (T-Test p<.001, Levene p=.020) and problem-solving competence (T-Test p=.011, Levene p=.019) show significant correlation between age and assessment.

Added Value: First, results reveal a ranking of needed competencies for the digital transition, which companies and educational institutions should address. Second, differences between the employee groups could be discovered which have to be considered in the further approach, be it education or research.

Relevance & Research Question: Using an online access panel of 10,000 households to precise target groups and their area-wide and microgeographical prediction for all regions and addresses

Methods & Data: The basis for the estimation is an online survey with 10,000 participants for which we have postal addresses. The addresses are crucial to match the microgeographic data from infas 360. The process of matching is called geocoding. Geocoding validates and locates addresses and integrates the individual geo key to which the microgeographical data are also attached. This database contains nearly 400 microgeographic variables to each of the 22 million unique addresses, e.g. building typology, building use, number of private households, average monthly net income, number of foreigners per block, unemployment rate per city district. Through the combination of the survey data and the microgeographic data, we localize, describe and analyze in a first step the responding households geographically, socio-demographically and according to other relevant characteristics. In a second step, we estimate the target group at 40 million households in Germany using a multi-level model. We demonstrate the procedure by means of an example, which is the prediction of the number of dogs per city district.

Results: Among other things the analysis in this example shows that dog owners live in houses with large gardens and are located mostly at the town border. Usually they live in regions with a high purchasing power and with a low migration share. This information is used to predict the number of dogs for all addresses with private households. Besides the quality criteria for the estimation model, we compare the results with official data from the Statistical Office Berlin on the level of city district. The comparison shows that the estimated data reflect the official data very well.

Added Value: Linking the online survey data with micro-geographical data from infas 360 enables a comprehensive description and segmentation of target groups as well as their address-specific transfer to the overall market as a prediction with astonishing precision. The use of an online access panel is cost efficient, flexible in time – and it is regionalizable.

Relevance & Research Question: Agenda-Setting research has been performed for more than four decades now as a matter of routine, in both traditional and online media. Recent years have seen a growing political use of social media messaging, especially during elections campaigns. The current study analyzes the effects of prime ministerial candidates' online messages on their followers' perceived agenda, as a function of the following patterns and the voting intentions, on which these followers have reported.

Methods & Data: To answer these questions, a representative sample of Israeli voters (n=1600) have answered a detailed Questionnaire. Questions regarded voting intentions, patterns of following prime ministerial candidates' accounts on social networking sites, and the followers perceived agenda.

Results: 43% reported that they follow a social network account of at least one prime minister candidate. 88% of them follow candidates through Facebook, 19% follow through Twitter and 18% through Instagram. The prominence of issues on the agenda differed significantly between those who reported that they followed candidates to those who did not follow them. Among the voters of the ruling party, 80% followed the party candidate exclusively, and 18% follow the party candidate in addition to the candidate of the leading opposition party. Among leading opposition party's voters, 48% follow the opposition candidate exclusively, and 47% also followed the ruling party candidate. The prominence of the issues on the agenda differed significantly between the exclusive followers of each of the two candidates.

Added Value:The present study indicates that social networking sites have a substantial impact on followers' perceived agenda during election campaigns. As social networks turn prominent than ever, it has become highly essential to deepen our understanding of their unique role in the political arena.

Relevance & Research Question: Frequently business decisions are relying on data collected through self-administered web surveys. The arguments derived from such datasets are based on non-probability convenience samples, provided by a self-selection of social media users completing online questionnaires. This kind of research is focusing on informative samples. Troubles caused by a lack of possibilities to test the representability are pushed into the background. However, projects rooted in such approaches have the potential to further increase undesirable respondent behaviors that include but may not be limited to speeding or straight lining. Not least because social media usage is connected to the users motivation, as it applies for survey participants. Based on this the present paper addresses two research questions: i) whether and to what extend does motivation (in connection with satisficing) influence the results of social-media surveys? ii) Are different effect detectable, caused by different accesses to the field?

Methods & Data: Analyses are based on three different data sets of social-media surveys. The first consists of 104 users of online forums for “gamers”, the second of 234 Facebook-users, which followed an invitation to evaluate the Facebook dialog with public utility companies and the third dealt with politics and was composed from a quota sample based on the Facebook network of students. In order to measure motivation and satisficing, individual indicators of motivation were included in the questionnaire, while satisficing strategies were measured by paradata (e.g. item-nonresponse, speeding). Correlation analyses and linear regressions were conducted.

Results: Indicators of motivation and of satisficing are just partly correlated. Unmotivated gamers are rather clicking though the survey (speeding), while on energy or politics interested, but unmotivated, Facebook-user, are rather causing item-nonresponses. The effect of motivation and satisficing on substantial results differs as well. Non-differentiation is of more importance following the results of gamers, while indicators of motivation has to be considered when the results of “unmotivated” Facebook-user are interpreted.

Added Value: The presentation raises awareness for the relevance of motivation and subsequently satisficing of social-media survey participants. Conducting such information is important in order to interpret the results of such informative samples appropriately.

Relevance & Research Question: Parallel to the technical development of mobile devices and smartphones, the interest in conducting ambulatory assessment studies has rapidly grown during the last decades. Participants of those studies are usually asked to repeatedly fill out short questionnaires. Besides numerous advantages, such as a reduced recall bias and a high ecological validity, the participant burden is higher than in cross-sectional studies or classical longitudinal studies. In our study, we experimentally manipulated participant burden (questionnaire length low vs. high) to investigate whether higher participant burden leads to lower data quality (e.g., as indicated by the compliance rate and careless responding indices). Moreover, we aimed to analyze effects of participant burden on the association between state extraversion and pleasant mood. We provided the questionnaires on two different platforms (questionnaire app vs. online questionnaire with link via e-mail) to investigate differences in the usability of those two presentation forms.

Methods & Data: Data were collected via online questionnaires and a smartphone application. After an initial online questionnaire (socio-demographic measures), participants were randomly assigned to one of four experimental groups (short vs. long questionnaires x app vs. online questionnaire). The ambulatory assessment phase lasted three weeks (one prompt per day in the evening). Participants rated the extent of situational characteristics, momentary mood, state personality, daily life satisfaction, depression, anxiety, stress, and subjective burden due to study participation. At the end of the study, participants filled out a short online questionnaire about their overall impression and technical issues. We computed the required sample size for mean differences (two-way ANOVA). 245 participants were needed to detect a small to medium effect (Φ = 0.18, power > 80%).

Results: Data collection is still running. The results of the preregistered hypotheses will be presented at the conference.

Added Value: This study can add insights into the effects of participant burden on data quality in ambulatory assessment studies. The results might serve as a basis for recommendations about the design of those studies. It would be desirable to conduct future ambulatory assessment studies in a way that participants are not overly stressed by their study participation.

Relevance & Research Question:

Accelerometers or actigraphs have long been a costly investment for measuring physical activity, but nowadays they have become much more affordable. Currently, they are used in many research projects, providing highly detailed, objectively measured sensory data. Where self-report data might miss everyday life active behaviors (e.g. walking to the shop, climbing stairs) accelerometer data provides a more complete picture of physical activity. The main objective of this research is identifying specific activity patterns using machine learning techniques and the secondary objective is improving the accuracy of identifying the specific activity patterns by validating activities through time-use data and survey data.

Methods & Data:

Activity data was collected through a large-scale accelerometer study in the probability-based Dutch LISS panel, consisting of 5.000 households. 1200 respondents participated in the study and wore a GeneActiv device for 8 days and nights, measuring physical activity 24/7. In addition, a diverse group of 20 people labeled specific activity patterns by wearing the device and performing the activities. These labeled data were used to train supervised machine-learning models (i.e. support vector machine, random forest) detecting specific activity patterns. A deep learning model was trained to enhance the detection of the activities. Moreover, 450 respondents from the accelerometer study also participated in a time-use study in the LISS panel. Respondents recorded their daily activities for two days (weekday and weekendday) on a smartphone, using a time-use app. The labeled activities were used to validate the predicted activities.

Results:

Activity patterns of specific activities (i.e. sleeping, sitting, walking, cycling, jogging, tooth brushing) were successfully identified using machine learning. The deep learning model increased predictive power to better distinguish between specific activities. The time-use data proved to be useful to further validate certain hard to identify activities (i.e. cycling).

Added Value:

We show how machine learning and deep learning can identify specific activity types from an accelerometer signal and how to validate activities by time-use data. Gaining insight in physical activity behavior can, for instance, be useful for health and activity research.

Relevance & Research Question: Citizen surveys as an initiating and innovative form of participation are becoming increasingly popular. They are advocated as a cost-effective and purposeful method of enhancing the public basis of the policy-making process, thus representing an appealing first step towards participation for local governments. With the rapid advance and increasing acceptance of the Internet it is now possible to reach a sufficiently large number of people from various population groups. However, in order to exploit these advantages to their full potential, it is important to gain insights into how to maximise the perceived impact and success for citizens. Ergo, how should local municipalities design and follow up on citizen surveys?

Methods & Data: We want to present key insights into citizen surveys as a participatory driving force based on more than twenty citizen surveys of various sizes and on various topics with over 12,000 participants in numerous municipalities (e.g. Alsfeld, Friedrichshafen, Konstanz, Marburg). More precisely, our main focus lies on the effective communication with the target population at the beginning of the process and the subsequent processing, visualisation and presentation of survey results.

Results: Citizen surveys can be used as an initiating process of enhancing political mobilisation and participation in the context of broader political processes, provided that rules and conditions are communicated early & clearly. Consulting citizens first but then deciding contrastingly is the worst imaginable approach, and yet this is still continuously occurring in practice. Key factors which contribute to the success of a survey include an objective evaluation, a thourough analysis and the usage of its results as future guidelines for policy-making.

Added Value: While citizen surveys are particularly well suited for initiating participation, it often remains unclear, how citizens perceive the impact their participation has and the overall success of the survey. Although there has been extensive research and debate about the specific design, the issues of preparating and following-up on citizens in order to promote responsiveness and efficiency, has – up until now – been widely. Accordingly, we seek to advance knowledge on these essential, yet scarcely studied, stages of implementation.

Relevance & Research Question: Increased survey completion time for mobile respondents’ completing web surveys is one of the most persistent findings. Furthermore, it is often used as a direct measure of cognitive load. However, as the measurement and interpretation of completion time faces various challenges, in our study we examined which possible sources of device differences ad to cognitive load, operationalized as completion time (objective indicator) as well as perceived cognitive load (subjective indicator). Furthermore, we wanted to examine whether cognitive load was functioning as a mediator between these sources and several data quality indices as proposed in the "Model of the impact of the mode of data collection on the data collected" by Tourangeau and colleagues.

Methods & Data: An extra questionnaire was added to our institutions mobile optimized, routinely used online course evaluation questionnaire. Key variables like distraction, multitasking, presence of others, attitude toward course evaluation in general as well as mobile device use were assessed. Additionally, paradata like device type and completion time were collected.

The sample consisted of participants of 107 mostly one-day continuing training courses for VET/PET professionals from the Italian-speaking part of Switzerland (N=1795).

Results: Consistent with previous research we found for mobile device use a self-selection bias and more reported distractions in the mobile completion situation as well as longer completion times and a higher perceived cognitive load. Several data quality indices like breakoff rate and item nonresponse were higher too, whereas straightlining was less. In addition, we found that the key variables in our study predicted the objective and subjective indicator of cognitive load differently and to a varying degree.

Added Value: The presented study suggests that cognitive load is a multifaceted construct. Its findings add to the existing limited knowledge on the question which survey factors are related to which aspect of cognitive load and how these in turn are related to different data quality indices.

Relevance and Research Question:

Longitudinal surveys allow researchers to study stability and change over time and to make statements about causal relationships. However, panel studies also hold methodological drawbacks, such as the threat of panel conditioning effects (PCE), which are defined as artificial changes over time due to repeated survey participation. Accordingly, researchers cannot differentiate “real” change in respondents’ attitudes, knowledge, and behavior from change that occurred solely as a result of prior survey participation which may undermine the results of their analyses. Therefore, a closer analysis of the existence and magnitude of PCE is crucial.

Methods and Data:

In the present research, we will investigate the existence and magnitude of PCE within the GESIS Panel - a probability-based mixed-mode access panel, administered bimonthly to a random sample of the German-speaking population aged 18+ years. To account for panel attrition, a refreshment sample was drawn in 2016. Due to the incorporation of the refreshment sample, it is possible to conduct between-subject comparisons for the different cohorts of the panel in order to identify PCE. We expect differences between the cohorts regarding response latencies, the extent of straightlining, the prevalence of don’t know-options, and the extent of socially desirable responding. Specifically, we expect that more experienced respondents show shorter response latencies due to previous reflection and familiarity with the answering process. Secondly, experienced respondents are expected to show more satisficing (straightlining, speeding, prevalence of don’t know-options). Finally, becoming familiar with the survey process might decrease the likelihood of socially desirable responding of experienced respondents.

Results:

Since this research is work in progress and related to a DFG-funded project which just started in December last year, we do not have results yet, but will present first results at the GOR conference in March.

Added value:

PCE can negatively affect the validity of widely used longitudinal surveys and thus, undermine the results of a multitude of analyses that are based on the respective panel data. Therefore, our findings will make a further contribution to the investigation of PCE on data quality and may encourage similar analyses with similar data sets in other countries.

Relevance & Research Question: The validity of surveys on sensitive issues is threatened by the influence of social desirability bias. Even in anonymous online surveys, some respondents try to make a good impression by responding in line with social norms rather than truthfully. This results in an underestimation of the prevalence of socially undesirable attitudes and behaviors. Indirect questioning techniques such as the Crosswise Model claim to control the influence of social desirability and to thereby increase the proportion of honest answers. The lower efficiency of indirect questioning techniques requires the use of larger samples that are more easily obtained online. We empirically investigated whether indirect questioning techniques indeed lead to more valid results.

Methods & Data: In a series of experiments we surveyed several thousand participants about different sensitive attitudes and behaviors. We randomly assigned the respondents to either a conventional direct questioning condition or an indirect questioning condition using the Crosswise Model. Prevalence estimates from the different conditions were compared using the “more is better” criterion. According to this criterion, higher estimates for socially undesirable attributes are potentially less distorted by the influence of social desirability and thus more valid.

Results: We found that higher and thus potentially more valid prevalence estimates could be obtained in the Crosswise Model conditions compared to the conventional direct questioning conditions. This finding shows that indirect questioning techniques were indeed capable of controlling the influence of social desirability and could thus increase the validity of the prevalence estimates obtained.

Added Value: Untruthful answers to questions on sensitive topics pose a serious threat to the validity of survey results. Our studies show that even in anonymous online surveys, the proportion of honest answers can be further increased through the use of indirect questioning techniques such as the Crosswise Model. Against this background, indirect questioning techniques appear as an effective means to overcome the harmful influence of social desirability bias on the results of online surveys.

Relevance & Research Question: Technical progress through digitalisation is constantly increasing. Currently, the most relevant and technically sophisticated technology is artificial intelligence (AI). Women are less frequently involved in research and development on AI, clearly in the minority in STEM-professions and study programmes, and less frequently in management positions. Previous AI applications have often been based on data that under-represents women and thus map our society with existing disadvantages and injustices.

So do men and women have different ideas about the role and significance of AI in the future? Do women have different requirements or wishes for AI?

Methods & Data: Following the previously conducted in-depth literature research, a combination of qualitative interview study [n=6] and quantitative online survey [n=200] is planned. The target group will consist of company representatives and students whereby the evaluation of differences and correlations will be based in particular on gender.

Results: A literature review of existing studies reveals that while more people are in favour of AI development than against it, it is mainly men with a high level of education and income. According to their self-awareness, women have a lower understanding of AI than men. Moreover, AI research and development is predominantly in the hands of men. Just under 25% of those employed in the AI sector are women, in Germany even only 16%. Old stereotypes are thus not only the basis for decisions regarding the development of AI but also incorporated into the data basis for AI: Voice and speech recognition systems are less reliable for female voices, as is face recognition for female faces. Search engines more often present male-connoted image and text results for gender-neutral search terms. The expected results of the questionnaire will be gender-relevant aspects in the perception, evaluation, development and use of AI.

Added Value: The identification of gender-relevant differences in the perception and attitude towards AI will enable developers and researchers to be sensitised to the possible risks of AI applications in terms of prejudice and discrimination. In addition, opportunities for using AI to strengthen gender equality will be recognized.

Relevance & Research Question: Weather is one unescapable environmental factor in human live. It significantly affects human behavior (e.g. daily activities), mood (e.g. helpfulness), well-being (ability to perform cognitive tasks) and communication (e.g. happiness of tweets). Human behavior, including communication, is also related to timing as another influencing factor, since days are determined by work-life routines (spending time at work), day-of-the-week (weekday vs. weekend) and holidays. The emotional status and mood is affected by the time-of-the-day and day-of-the-week (e.g. blue Monday), too. In further consequence, it cannot be excluded that weather and timing influences online communication. Therefore, this research aims to show the differences in the nature of data of social media communication due to the time of data collection - in connection with the then prevailing weather conditions.

Methods & Data: 321 postings, published at a public utilities official Facebook-account in the time-frame between August 2016 and February 2018 are taken into account. The influence of weather and timing on the posting content (company-relation), the posting visualization, the posting length and subsequently on the stakeholders’ reactions (number), comments (yes/no) and shares (yes/no) are examined by means of multivariate analyses.

Results: Temperature and barometric pressure are influencing – quite consistent – the company’s released postings, while precipitation and humidity is more decisive for the users reply to a post. The timing, on the other hand, shows rather unstable influences. Nevertheless, both factors influence the virtual communication and the data quality in social media research.

Added Value: The presentation raises awareness for the relevance of weather and timing - when investigating the behavior in social media, researchers need to be aware of the effect of both.

Relevance & Research Question:

Online fundraising is becoming increasingly important for non-profit organisations, but the

factors that convince people to make a donation online have not yet been fully investigated.

Methods & Data:

In the present work, data of two studies (total N = 2525) was used to examine factors of online donation. Probands completed an online survey, where they rated the design of specific websites as well as gave a statement about a possible donation to this website.

An effect of website design (as content design, usability and aesthetics) on willingness to donate was postulated. Furthermore, research questions about demographic aspects such as age and gender as well as trust in the organization were posed. For statistical analysis, logistic regressions were performed.

Results:

The results showed different predictors of donation in different scenarios of donation.

For the donation of one's own money, the perceived content (Odds Ratio = 1.99) and trust in the organization (OR = 1.95) showed the greatest correlations.The usability, on the other hand, showed a negative correlation to this, the effect strength was clearly smaller here (OR = 0.76). In this model, the perceived aesthetics of the website showed no significant correlation to the dependent variable.

When donating other people's money, the aesthetics of the website was the most important factor for the willingness to donate (OR = 1.35). The logistic regression showed that the majority of predictors had no significant relationship to the dependent variable.

The demographic variables showed different correlations between the studies.

Added Value:

The relevance of the design of the website, but also of trust in the organization, was shown, from which the first implications for online fundraising can be derived. The effects seem to vary for different scenarios of online donation. The differences between the two scenarios can be explained by the increased relevance of the decision, which results from donating one's own money. Therefore, more factors are included and reflected in such a decision. Regarding this, further research is necessary, examining influences of other variables and establishing implications for successful digital donation generation in the healthcare sector.

Establishing a digital-connection with our customers on a superficial level is not hard. Chatbots have been around for years and however that’s not really a deep connection. Consumers in most industries expect significantly higher levels of personalisation of our products and services. But mass-personalisation requires companies to bring together and analyse huge volumes of data that the Market Research Industry is not used to analysing. So what’s going to be our role in this new world?

Client Business Intelligence (BI) and Advanced Analytics departments all around the world are trying to align and harmonise badly-structured data. And they are not calling market research traditional firms to do it. Are they using AI? Not yet, but will they?

What’s the state of the art of AI in other industries and how is that relevant to the world of MR? I invite you to brainstorm together.

Patricio Pagani is the founder of The Black Puma Ai, a company dedicated to blending the power of human and digital brains to augment organizational intelligence. Patricio is also a Digital Transformation catalyst that is helping large corporations embrace what the future holds for them.
An Angel Investor in technology startups (mobility, IOT), Patricio has a portfolio of companies he's advising. Also, he is a Board director at Infotools, a leading provider of market research software tools & services.
A sought-after keynote speaker at various marketing forums, you will find Patricio discussing what the future may hold for the Business Intelligence and Market Research industry. Patricio used to be the president of the New Zealand Market Research Society for several years and is currently the ESOMAR representative for Argentina.

Relevance & Research Question: In videomediated (Skype) survey interviews, how will the small self-view window affect people's disclosure of sensitive information and self-reported feelings of comfort during the interviews? This study replicates and expands on previous research by (a) tracking where video survey respondents look on the screen—at the interviewer, at the self-view, or elsewhere—while answering questions and (b) examining how gaze location and duration differ for sensitive vs. nonsensitive questions and for more and less socially desirable answers.

Methods & Data: In a laboratory experiment, 133 respondents answered sensitive questions (e.g. sexual behaviors) and nonsensitive questions (e.g. reading novels) taken from large scale US government and social scientific surveys over Skype, either with or without a self-view window. Respondents were randomly assigned to having a self-view or not, and interviewers were unaware of the self-view manipulation. Measures of gaze were recorded using an unobtrusive eye-tracking system.

Results: The results show that respondents who could see themselves looked more at the interviewer during question-answer sequences about sensitive (compared to nonsensitive) questions, while respondents without a self-view window did not. Respondents who looked more at the self-view window reported feeling less self-conscious and less worried about how they presented to the interviewer during the interview. Additionally, the self-view window increased disclosure for a subset of sensitive questions, specifically, total number of sex partners and frequency of alcohol use. Respondents who could see themselves reported perceiving the interviewer as more empathic, and reported having thought more about what they said (arguably reflecting increased self-awareness). For all respondents, gaze aversion—looking away from the screen entirely—was linked to sensitive (or socially undesirable) responses and self-presentation concerns.

Added Value: Together, the findings demonstrate that gaze patterns in videomediated interviews can be informative about respondents’ experience and their response processes. The promise is that findings like these can contribute to the design of new, potentially cost-saving video-based data collection interfaces. This study also provides necessary groundwork for continued investigation not only of mode effects on disclosure in surveys (as one measure of response accuracy) but also on interactive discourse more generally.

Relevance & Research Question: So far, a number of studies have examined nonprobability and probability-based panels, but mostly only with regard to survey sample accuracy. In this presentation, we compare nonprobability and probability-based panels on a new dimension: we examine what happens when panel members are asked to use a mobile app to record their spending. We answer the following research questions: Do different types of people participate in the app study? Are there differences in how participants use the app? Do differences between samples matter for key outcomes? And do differences between samples remain after weighting?

Methods & Data: To answer our research questions, we use data from Spending Study 2, which is an app study that was implemented in May to December 2018 in two different panels in Great Britain: Understanding Society Innovation Panel, which is a probability-based panel, and Lightspeed UK, which is a nonprobability online access panel. In both panels, participants were asked to download a mobile app and use it for one month to report their spending. In our presentation, we compare the app data collected from the participants of the two panels.

Results: Our analyses show that different people participate in the app study implemented in the nonprobability and probability-based panel, both in terms of socio-demographic characteristics and with regard to digital affinity and financial behavior. Furthermore, the app study leads to different conclusions in terms of key substantive outcomes, such as the total amount and type of spending. Moreover, differences between the app study samples on substantive variables remain after weighting for socio-demographic characteristics. Only the way in which the app study participants use the app does not seem to differ between the panels.

Added Value: Our study contributes to the ongoing discussion on nonprobability and probability-based panels by adding new empirical evidence. Moreover, our study is the first to examine app study data rather than survey data. Furthermore, it covers a wide range of data quality aspects, including sample accuracy, respondent participation behavior, and weighting procedures. We thereby contribute to widening the debate to non-survey data and multi-dimensional data quality assessments.

Relevance & Research Question: Mobile devices are designed to be used while people are on the move. In the context of a mobile web survey, researchers should consider the potential consequences of respondent mobility for data quality. Being exposed to sources of distraction could result in suboptimal answers and an increased risk of breakoff. This study investigates whether there are between-device differences (web and mobile web browser vs. smartphone app) in terms of the context in which questionnaires are completed. We consider: 1) day, time and location of survey participation; 2) whether participants’ location changes during completion; and 3) whether differences in completion context are related to breakoffs and item nonresponse.

Methods & Data: We use data from an experiment embedded in a three-wave probability, general population survey conducted in Switzerland in 2019 (N=2,000). Half the sample was assigned to an app-based survey; the other half to a browser-based survey, encouraging mobile web completion. We use a combination of questionnaire data (on current location), paradata (timestamps and location indicators), and respondents’ photos of their surroundings taken at the beginning and end of the survey to gain insight into completion conditions.

Results: Our results suggest a minority of respondents was ‘on the move’ while filling out the survey questionnaires. Mobile web browser users were more likely to answer in the evening, while PC browser users responded in the late afternoon. Photographs indicate that app users tended to complete the survey at home, although the app was designed to be used on the move (using a modular design with questionnaire chunks which took less than three minutes). Furthermore, app users were unwilling to move outside to complete a different photo task.

Added value: The findings inform the design of mobile web surveys, providing insights into ways to optimise data collection protocols (e.g. by tailoring the timing of survey requests in a mixed device panel design), and to improve the onboarding procedure for smartphone app respondents. The provision of unique log-in credentials may have inhibited participant mobility and the possibility to take advantage of this key feature of mobile internet technology.

Relevance & Research Question: In social science research, questions with unipolar and bipolar scales are commonly used methods in measuring respondents’ attitudes and opinions. Compared to other rating scale characteristics, such as scale direction and length, scale polarity (unipolar and bipolar) and its effects on response behavior have been rarely addressed in previous research. To fill this gap in the survey literature, we investigate whether and to what extent fully verbalized unipolar and bipolar scales influence response behavior by analyzing observed and latent response distributions and latent thresholds of response categories.

Methods & Data: For this purpose, we conducted a survey experiment in the probability-based German Internet Panel (N = 2,427) in March 2019 and randomly assigned respondents to one of the following two groups: the first group received four questions on income (in)equality with a five-point, fully verbalized unipolar scale (i.e., agree strongly, agree somewhat, agree moderately, agree hardly, agree not at all). The second group received the same four questions on income (in)equality with a five-point, fully verbalized bipolar scale (i.e., agree strongly, agree somewhat, neither agree nor disagree, disagree somewhat, disagree strongly).

Results: The results reveal substantial differences between the two rating scales. They show significantly different response distributions and measurement non-invariance. In addition, response categories (and latent thresholds) of unipolar and bipolar scales are not equally distributed. The findings show that responses to questions with unipolar and bipolar scales differ not only on the observational level, but also on the latent level.

Added Value: Both rating scales vary with respect to their measurement properties, so that the responses obtained using each scale are not easily comparable. We therefore recommend not considering unipolar and bipolar scales as interchangeable.

Relevance & Research Question: Designing grid questions for smartphone surveys is challenging due to their complexity and potential increase in response burden. This paper comprehensively reviews the findings of scientific studies on several data quality and response behavior indicators for grid questions in smartphone web surveys: satisficing, missing data, social desirability, measurement quality, multitasking, response times, subjective survey evaluation, and comparability between devices. This framework is used to discuss different grid question design approaches and their data quality implications.

Methods & Data: Experimental studies investigating grids in smartphone surveys were identified using the DiKUL bibliographic harvester that includes over 135 bibliographic databases. The string “’mobile web survey’ AND (grid OR scale OR matrix or table)” returned 55 results. After full-text evaluation, 35 papers published in English between 2012 and 2018 were found eligible for extraction of findings regarding the eight groups of data quality and response behavior indicators.

Results: Grid questions tend to increase self-reported burden and satisficing behavior. The incidence of missing data increases with the number of items per page and per grid. The comparisons between smartphones and PCs yield largely mixed results. While completion times are decisively longer on smartphones, the grid format has little to no effect on response times compared to item-by-item presentation. Differences in satisficing are modest and observations about the relationship between missing data and device type are mixed. No effects of device type on socially desirable responding were detected, while differences in measurement quality are mostly limited to worse input accuracy and biased estimates on smartphones due to noncoverage and nonresponse error. Mixed are also the findings about differences in multitasking.

Added Value: Data quality remains a salient issue in web surveys, as well as in the context of the visual syntax that defines the design of survey questions on different devices. This review of current practice offers insights into data quality implications of the principles for designing grid questions in smartphone web surveys and their comparability to web questionnaires on PCs. The critical elaboration of findings also provides a guidance for future experimental research and usability evaluation of web questionnaires.

Relevance & Research Question: In survey research item nonresponse is regarded as an important criterion for data quality among other quality indicators (e.g. breakoff rate, straightlining etc.) (Blasius & Thiessen, 2012). This originates from the fact that, as with the unit nonresponse rate, persons who do not answer a specific item can systematically differ from those who do. In online surveys this threat can be countered by using the possibility of prompting after item non-response. In this case prompting means a friendly reminder displayed to the respondent, uniquely inviting him to give his answer. If the respondent does not want to answer it is possible to move on in the questionnaire (soft prompt). The forced choice option however requires a response on every item.

There is still a research gap on the effects of prompting or forced choice options in web surveys on data quality. Tourangeau (2013) also comes to the following conclusion: ‘More research is needed, especially on the potential trade-off between missing data and the quality of the responses when answers are required’.

Methods & Data: We conducted a methodological experiment using a non-probability sample recruited over a social network platform in January 2019. To test the different prompting options we implemented three experimental groups (forced choice, soft prompt, no prompt) (Total N = 1,200). Besides item-nonresponse rate we used the following data quality indicators: breakoff rate, straightlining behavior, duration time, tendency to give social desirable answers and self-reported interest.

Results:The results show a higher breakoff rate for specific questions where forced choice was applied. Furthermore a higher item nonresponse rate was found for the no prompt option. Overall only small effects were found. Final results on the different effects on the data quality will be presented at the conference.

Added Value: We found little to no evidence on the impact of prompt options on data quality. However, we found that soft prompt tends to lead to lower item nonresponse compared to no prompt.

Relevance & Research Question: Smartphones make digital media and other digital means of communication constantly available to individuals. This constant availability may have a significant impact on individuals exhaustion levels. In addition, being available often brings social pressure (e.g., "telepressure") at work or at home that may lead to a further increase in exhaustion at the end of the day. On the other side, constant connectivity may enable frequent contact to one’s social networks what might decrease exhaustion. We examine whether employees perceive "being available" as a burden or as a resource in their daily work.

Methods & Data: We use a combination of data from a probability based population panel from Germany (Panel Study Labour Market and Social Security -- PASS) and a research app (IAB-SMART), which passively collected smartphone data (e.g. location, app usage) and administered short daily surveys. Since app participants (N=651) were recruited from PASS, we are able to link both data sources. The PASS data provides us with sociodemographic variables, e.g. age, education, gender etc. and background information, which enables us to calculate population weights. From the passively collected app data, we can construct a series of predictors like daily smartphone usage and instant switches between apps. The level of exhaustion is measured by a survey question, which was daily repeated for seven days every three months, i.e., we have one to 14 measures per individual. Considering several selection processes within the data collection, we end up with an analysis sample of 163 individuals with 693 days that we use in a multilevel regression model.

Results: Our analysis is in an early stage. Therefore, we are not able to share results at the time of submitting this abstract.

Added Value: First, we assess if and how daily smartphone usage correlate with levels of exhaustion for individuals. Second, our analysis shows how a combination of survey and passive data can be used to answer a substantial question. Third, we share our experience of how to feature engineer variables from unstructured mobile phone data to valid variables that may be used in a variety of field in general online research.

Relevance & Research Question:

Smartphone apps are starting to be commonly used to measure travel behaviour. The advantage of smartphone apps is that they can use location sensors in mobile phones to keep track of where people go at what time at relatively high precision. In this presentation, we report on a large fieldwork test conducted by Statistics Netherlands and Utrecht University in November 2018 and present on the quality of travel data using hybrid estimation using passive data and a diary-style smartphone app.

Methods & Data: A random sample of about 1900 individuals from the Dutch population register was invited by letter to install an app on their smartphone for a week. The app then tracked people's location for a week continuously. Based on an algorithm the app divided each day into “stops” and “tracks” (trips), which were fed back to respondents in a diary-style list separately for every day. Respondents were then asked to provide details on stpops and trips in the diary.

Results:

Having both sensor data and survey data allows us to investigate measurement error in stops, trips and details about these in some detail. A few types of errors may occur:

1) False positives: a stop was presented to a respondent that wasn’t a stop (and by definition also a track connecting this stop to another one).

2) False negatives: stops were missing from the diary (often because a respondent forgot the phone, or GPS tracking was not working properly).

How can we identify false positives and negative? How did respondents react to false positives, and how can we correct for this in estimates of travel behaviour?

Added Value: We will discuss each type of error, their size,and the context in which they occurred. Finally, we will discuss the overall impact of both false positive and false negatives and discuss their overall impact on the statistics of interest. We conclude with a discussion of how to generally move forward in combining sensor and survey data for tracking studies for social science, market research and official statistics

Relevance & Research Question: ---Early studies investigating willingness to participate in surveys involving smartphone data collection apps – and particularly, to consent to passive data collection – have identified concerns relating to data privacy and the security of shared personal data as an important explanatory variable. This raises important practical and theoretical challenges for survey methodologists about how best to design app-based studies in a way that fosters trust and the implications for data quality. We address the following research questions: 1) How do data privacy concerns vary among population subgroups, and as a function of internet and smartphone usage habits? 2) To what extent do expressed data privacy concerns predict stated and actual willingness to participate in an app-based survey involving passive data collection?---

Methods & Data: ---The data were collected in an experiment embedded in a three-wave probability-based, general population election study conducted in Switzerland in 2019. At wave 1, half the sample was assigned to an app-based survey, and the other half to a browser-based survey; at wave 2, the browser-based respondents were invited to switch to the app. At wave 1, respondents in both groups were asked about their attitudes to sharing different types of data and about their data privacy and security concerns. The quantitative findings are complemented with findings from user experience research.---

Results: ---Consistent with other studies, preliminary results show statistical differences in levels of concern about data privacy and the degree of comfort sharing different data types across subgroups (e.g. based on age, sex and response device) and confirm that privacy concerns are an important predictor of actual participation in a survey using an app.--

Added Value: ---Given the often weak relationship between attitudes and behaviours, and the apparent paradox between privacy attitudes and actual online data sharing behaviours, the possibility to assess how data privacy concerns affect actual participation in an app-based study of the general population is of great value. We propose avenues for future research seeking to reduce public resistance to participate in smartphone surveys involving both active and passive data collection.---

Relevance & Research Question: This study is the first in the American context to compare political candidates’ social media communication across multiple social media platforms. This topic is relevant, as the large majority of digital political communication studies only focus on one social media platform. We therefore ask two research questions:

RQ1: Do political campaigns broadcast the same messages across multiple social media accounts, or does campaign messaging differ depending on the platform?

RQ2: What explains the similarity or difference in political campaigning across social media platforms?

Methods & Data: We combine three types of computational analysis – fuzzy string matching, automated content analysis, and machine learning classification – to compare the Facebook and Twitter posts of Hillary Clinton and Donald Trump during the 2016 U.S. Election.

Results: Our results show a relatively high degree of content recycling across platforms. At the highest level, over 60% of Clinton’s Facebook posts were also present on Twitter, whereas approximately 1/4 of the Trump campaigns posts were recycled across the two platforms. We do, however, find key strategic differences relating to how this content was conveyed to electorate. Our machine learning algorithm categorized posts by topic issues and message type, and we found the latter to be a significant predictor of platform differentiation through chi-squared tests. That is, candidates promoted the same policy issues across platforms, but the strategic intent behind their messages differed. Most notably, the Clinton campaign messaged Hispanic audiences in Spanish solely on Facebook. The Trump campaign promoted livestreams predominantly on Facebook, while reserving Twitter for broadcasting information relating to mass media interviews.

Added Value: The added value of the study is two-fold. First, while the state-of-the-art suggests candidates use different platforms for different messaging, we find a relatively high degree of content recycling across platforms. Moreover, we go beyond the existing literature and uncover what explains differences in cross-platforms posts. It is not the policy content of messages, but rather the strategic motivations that campaigns perceive in light of the audiences on each social media platform.

Relevance and research question: During more then a decade now, political candidates have been using central social networks as their leading platforms in election campaigns, constantly trying to improve their performance and enhance their influence over potential voters. In 2019 only, Israel has seen two general elections. Comparing patterns of online platforms' political usage between these two campaigns reveals some surprising changes: within a few months, innovative components that were used in the first election campaign were abandoned in the second (e.i. using live television broadcasts on candidates' Facebook accounts). Could these changes indicate a broader phenomenon? Can we detect evidence of a decline in exposure to innovative platforms already in the first campaign?

Methods and data: Four consecutive surveys were passed during the last month before the first 2019 Israeli general elections, with 520-542 respondents participated in each. The samples represented the Israeli voters' population and were transmitted via an online panel to match the actual distribution of the population. The questionnaire included 50 questions on voting trends, news exposure patterns, and political content exposure in traditional and new media, most of them on a Likert scale.

Results: A One-Way Analysis of Variance was conducted to examine overall exposure to online political content. No significant difference was found at the four-time points examined [F (3, 2153) =0.271, p> 0.05]. More specifically, a Kruskal-Wallis test was performed to examine whether there was a statistically significant difference in the use of various social media applications. There were no significant differences in the consumption of Facebook, Twitter, and Telegram apps; however, a change in exposure to political content was detected on Instagram [Kruskal-Wallis H = 9.42, df=3, p < 0.05] with decreased of the mean rank score.

Added value: The findings of the current study suggest that politicians' attempts to be innovative in online media are not necessarily effective. Despite politicians' efforts, there is no detectable increase in exposure to new platforms as Election Day approaches.

Relevance & Research Question:

What are the effects of campaigns on voting behavior? Despite a long tradition in research on the question we know surprisingly little about it. A main reason is that the analysis of polls of polls—the most important method for the comparative analysis of campaign effects—is confronted with formidable methodological problems. In this paper, I propose and apply an alternative method for the analysis of campaign effects on voting behavior, which is based on prediction markets.

Methods & Data:

The proposed method to analyze campaign effects in elections and direct democratic votes is based on real money online prediction markets with automatic market scoring rules. In contrast to the current use of prediction markets, I propose to let traders bet on the probabilities according to which sequences of vote shares will be the outcome of a vote. I use original data that I have collected in the context of 16 direct democratic votes in Switzerland.

Results:

I demonstrate that on average direct democratic campaigns in Switzerland have substantial mobilizing effects and make up for about 5% of Yes-vote shares. Furthermore, campaign effects vary in predictable ways between ballots.

Added Value:

I illustrate this "sequencing method" with the first time–series cross–section analysis of direct democratic campaigns. Since this is the first paper to quantify the total effect of campaigns for direct democratic decisions it has a major impact on our understanding of direct democracy.

The important role of emotions in advertising is undoubted and researchers are experimenting with different ways to measure emotion. According to linguist scientist Sendlmeier “voice conveys our emotional state of mind in the most differentiated way”. With digital speech assistant technology and artificial intelligence new opportunities for research arise.

In the study voice data collection followed up by a high-end automatized emotion data analytics process was implemented in an online research design. The goal of this approach is a) to examine whether this approach is implementable from a technological point of view (realization)

b) to critically analyze the emotion analytic outcome from a research perspective (validation).

An online-representative sample was recruited: participants were confronted with an experimental setting of online video campaigns, audio ads and Instagram ads. After completing a classic item battery for advertising research, participants had to answer questions with their microphone from the device they were using. The device agnostic approach allows to include Desktop, Laptop, Tablet and Smartphone in the research design. The audio data is analyzed with two API AI approaches:

a) automatic transcription from voice into text (“what”)

b) emotional analysis of the tonality of the voice (“how”).

As results the approach offers the content and the analysis of 21 emotional facettes out of the audiofile for each participant.

This research design was effective from a technical perspective: 859 participants (out of 3760 starters) could be analyzed including emotional profiles from each answer. The voice analytic approach shows interesting divergence from the classic answer patterns. Further research approaches should focus in detail on the validity of the emotional scores from audio interactions.

The study combined for the first time an automatized data collection and analytics API approach of voice data with automation in transcription and emotional impact measurement. The promising results clearly show the power of artificial intelligence driven research approaches which will change the landscape of research very soon.

Relevance & Research Question:

Since the market entry of the WESTbahn in December 2011, Austrian rail passengers have the choice been between two railway companies on the Western route (Vienna - Salzburg). WESTbahn pursues among others the goal of attracting long-term customers through a very good range of rail services at very low prices (Koroschetz 2014). In this context, the question arises whether further growth in demand is realistic even if WESTbahn better meets customers' willingness to pay thanks to a differentiated ticket structure.

Methods & Data:

In order to ensure a holistic market and customer perspective, different empirical studies were conducted in 2019 and later linked together: First. a representative study focusing on travelers on the Western Line, second a customer survey (offline) during the train journey, and third a survey of ticket buyers on westbahn.at. Secondary data and information of sales and revenue management systems were used to validate the survey results.

Results:

While the market study supports the hypothesis that the railways as a whole have growth potential on the Western Line, it has become apparent that the price as a determinant plays a central role for future growth. To understand the opportunities for shifts in demand within the rail system, customer segmentation is essential, which describes the affinity of rail customers for WESTbahn and OEBB. In the case of the existing WESTbahn customers, there was a considerable spread in willingness to pay, which could be used to differentiate the ticket structure.

Added Value:

Since the fall of 2018, several changes have been made in WESTbahn's price and revenue management (prices have been partly raised, partly reduced, resulting in a stronger price differentiation). Customer surveys supported the project at all stages (conception, testing, implementation, monitoring). As a result, WESTbahn not only continued to grow through demand gains, but also achieved a change in the ticket mix, price levels and double-digit sales growth. At the same time, WESTbahn achieves top marks in terms of customer satisfaction and the intention to recommend.

Relevance & Research Question: Due to growing environmental requirements, increasing autonomy and changes in mobility behavior, the automotive industry is facing great challenges. Suppliers are under pressure to identify and implement new developments and customer requirements at an early stage in order to keep up with the highly competitive automotive market.

One of the areas in which Asahi Kasei specializes are surfaces and acoustics for vehicle interiors. SKOPOS supports them in establishing which requirements and wishes customers will have regarding vehicle interiors of the future resulting from the changing role of car sharing and autonomous driving.

Answering these questions confronted us with the challenges of not limiting the participants in their thinking about future developments whilst simultaneously leading them towards products from the Asahi Kasei product range.

Methods & Data: We chose a standard quantitative online approach - but with a twist. We presented car drivers and those open to car sharing with a primarily quantitative questionnaire, assessing their mobility behavior and the evaluation of (future) car features regarding their own and shared cars. The twist: We also implemented open questions based on our experience with online research communities, to maximize involvement and effort leading to better and more creative output.

Results: Cleanliness inside the car, especially in car sharing, is the most important factor when it comes to the interior of a car. The overall usability of features within the interior of a car is more important than premium surfaces. Finally, participants responded to open questions with longer and more extensive answers compared to similar automotive studies.

Added Value: The holistic and individualistic approach we followed with our research has proven to be very useful for Asahi Kasei and their business problem. Beginning with the development of the questionnaire, the analysis of the collected data and finally the consultation based on the results: Despite limited budget and time constraints, we were able to lay the groundwork for Asahi Kasei’s future developments of car interiors, enabling them to present the results to current and future customers. In addition, we successfully introduced them to the wonderful world of market research!