Conference Agenda

Overview and details of the sessions of this conference. Please select a date or location to show only sessions at that day or location. Please select a single session for detailed view (with abstracts and downloads if available).

Session Overview
Date: Wednesday, 06/Mar/2019
9:00 - 10:00Begin Check-In
10:00 - 1:00Workshop 1
Room 158 

Developing Dashboards in Microsoft Power BI

Gernot Heisenberg

TH Köln, Germany

Duration of the workshop: 2.5 h

Target groups: all data-affine people

Is the workshop geared at an exclusively German or an international audience? international

Workshop language: English (German if no international attendees)

Description of the content of the workshop:

  1. Getting started with Power BI.
  2. Read data and do preprocessing using the Power BI Editor.
  3. Perform simple to complex data transformations.
  4. Visualize data using Power BI visuals and how to select appropriate ones.
  5. Get more out of your data using the DAX expression language in Power BI.
  6. Tell a story with your data.
  7. Upload and publish your dashboard.

Goals of the workshop:

You will have to basics for analyzing and visualizing data in an interactive dashboard using Power BI.

Necessary prior knowledge of participants:

Excel knowhow is crucial and to have a good feeling for data.

Literature that participants need to read prior to participation: none

Recommended additional literature: "Power BI from Rookie to Rock Star" by Reza Rad

Information about the instructor:

Maximum number of participants: 20-30

Will participants need to bring their own devices in order to be able to access the Internet? Will they need to bring anything else to the workshop? Yes, please bring your own notebook (Windows or Mac with Paralles/Dual Boot). Please add a mouse, since we are going to click a lot :-)

10:00 - 1:00Workshop 2
Room 149 

The Questback Data Privacy Assistant

Hannah Esser

Questback GmbH, Germany

Duration of the workshop: 2.5 h

Target groups: online researchers in general

Is the workshop geared at an exclusively German or an international audience? international

Workshop language: English

Description of the content of the workshop: This workshop will provide an introduction to the data privacy assistant in EFS - a tool that is designed to aid in the GDPR compliance of surveys. In this workshop participants will learn about the different features of the tool, their relevance to GDPR as well as their basic implementation in an EFS project.

Goals of the workshop: Basic understanding of how the Questback software can support in conducting GDPR compliant surveys

Necessary prior knowledge of participants: Basic EFS knowledge, no prior knowledge of the data privacy assistant

Literature that participants need to read prior to participation: none

Recommended additional literature: none

Information about the instructor: As consultant at Questback, Hannah Esser predominantely works in the HR sector implementing large employee surveys grounded in complex organizational structures. She has a background in Psychology and marketing and a well-founded knowledge in empirical research methods.

Maximum number of participants: 8

Will participants need to bring their own devices in order to be able to access the Internet? Will they need to bring anything else to the workshop? Participants need to bring their own devices, Internet access is required and will be provided by DGOF.

1:00 - 2:00Lunch Break
2:00 - 5:00Workshop 3
Room 147 

Smartphones: From Survey Design to Sensor Data

Vera Toepoel, Anne Elevelt

Utrecht University, Netherlands, The

Duration of the workshop: 2.5 h

Target groups: people involved in conducting surveys

Is the workshop geared at an exclusively German or an international audience?

Workshop language: English

Description of the content of the workshop:
The release of the first iPhone was now more than a decade ago and smartphones have since become a mainstream device. In many countries, smartphones are replacing traditional PCs and laptops as the primary device to browse the Internet and to use social media. In the last couple of years, researchers have experimented with smartphones as a method of data collection. This short course focuses on recent studies that have aimed to study how smartphones can be used. 1. As a device to administer surveys and 2. to acquire additional behavioral data using sensors.

Goals of the workshop:
Understand why you should want to do research using smartphones; learn how to make web surveys mobile-friendly, understand issues around the collection of smartphone sensor data

Necessary prior knowledge of participants:
No previous knowledge is required, although an understanding of survey methods (the TSE framework, questionnaire design) will be helpful.

Literature that participants need to read prior to participation: none

Recommended additional literature: none

Information about the instructor:

Vera Toepoel is an assistant professor in survey methodology at the Department of Methods and Statistics at Utrecht University, the Netherlands. Her research interest lie in everything related to survey methodology and online surveys in particular: from recruiting respondents, designing the survey instrument, correcting for bias etc. Current topics include data chunking (a.k.a. modular survey design), sensor data (and consent) and mobile survey design. Vera is the President for RC33 (Methods and Logistics) of the International Sociological Association. She is a member of the Scientific Quality Assurance Board of the GESIS Online Panel in Germany. Vera is the author of the book “Doing Surveys Online” published by Sage (2016), has authored several chapters in handbooks for methodology, and has published numerous journal papers amongst others in Public Opinion Quarterly, Sociological Methods and Research, Survey Research Methods, Social Science Computer Review, Survey Practice etc.

Anne Elevelt is a PhD student at the Department of Methods and Statistics at Utrecht University. Her PhD thesis focuses on smartphone surveys and sensor data.

Maximum number of participants: no

Will participants need to bring their own devices in order to be able to access the Internet? Will they need to bring anything else to the workshop? no

2:00 - 5:00Workshop 4
Room 149 

The Meta-Analytical Research Process

Jessica Daikeler, Bernd Weiß

GESIS Leibniz Institute for the Social Sciences, Germany

Duration of the workshop: 2.5 h

Target groups:
The workshop is aimed at researchers planning their own meta-analysis or evaluating the quality of meta-analyses.

Is the workshop geared at an exclusively German or an international audience? international audience

Workshop language: English

Description of the content of the workshop:
This workshop will provide an overview of the individual steps of a meta-analysis (problem statement, systematic retrieval of studies, coding/ transformation, analysis, and interpretation) and will place particular emphasis on quality aspects.

Goals of the workshop:

  • get to know the specific steps of a meta-analysis
  • best pratices and evaluation in meta-analystical techniques"

Necessary prior knowledge of participants:
Being familar with published meta-analysis. i) Literature that participants need to read prior to participation Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. John Wiley and Sons. (Preface, part 1; Pages xxi-xxviii and 3-14)

Recommended additional literature: none

Information about the instructor:

Jessica Daikeler is currently preparing her dissertation on "The application of meta-analyses in survey methods". She is working on meta-analytical publications on response rate comparison, interviewer training, and paradata.
Dr. Bernd Weiß is head of the GESIS Panel, and deputy head of the GESIS department Survey Design and Methodology. He currently serves as co-chair of the Campbell Collaboration training group. He is currently involved in systematic reviews and meta-analyses in the areas of survey methodology, family sociology, economics as well as educational sciences.

Maximum number of participants: 20

Will participants need to bring their own devices in order to be able to access the Internet? Will they need to bring anything else to the workshop? no

2:00 - 5:00Workshop 5
Room 158 

Predicting Online Behaviour

Denis Bonnay

respondi & Université Paris Nanterre, France

Duration of the workshop: 2.5 h
Target groups: data scientists and market researchers interested in browsing data and web behaviour and / or machine learning application to market research.
Is the workshop geared at an exclusively German or an international audience? International
Workshop language: English
Description of the content of the workshop:
Cookies, tracking devices and passive measurement tools provide us with a host of new data regarding people's online behaviour. However big that data is, we often find ourselves in need for going beyond the data in order to predict browsing we did not actually observe. Predictive models of online behaviour are useful in a variety of contexts, such as 1/ when we need to generalize results from a tracked sample to a non-tracked sample (e.g. in order to provide information about online habits for survey respondents or for behavioural sampling), 2/ when we need to feel gaps in online journeys (e.g. because cookies only provide us with partial information), 3/ when we need to explain collective browsing behaviour and not simply to describe it (assuming that a good predictive model also has explanatory value).
The workshop will focus on two predictive tasks, particularly relevant to 1/ and 3/ above: prediction of website visits on the basis of sociodemographic and declarative internet usage data by means of nearest neighbour methods and prediction of news consumption with specific topics by means of generative models.
Goals of the workshop:
The goal is to promote the use of machine learning on passive data as a supplement to declarative data, by helping market researchers figure out what they can ask their data scientists, and by sharing with data scientists some ideas and methods to make the best out of this kind of data.
Necessary prior knowledge of participants:
The workshop will be aimed at diverse communities, from data scientists to market researchers and will not require any specific prior knowledge.
Literature that participants need to read prior to participation: None
Recommended additional literature:
Gleeson, J.P., Cellai, D. & alii “A Simple Generative Model of Collective Online Behavior”, Proceedings of the National Academy of Sciences, 111 (29), 10411-10415, 2014.
Van den Poel, D. & Buckinx, W. “Predicting online-purchasing behaviour”, European Journal of Operational Research, 166 (2), 2003.
Information about the instructor:
Denis Bonnay is in charge of data science at Respondi, where he is dedicated to developing research methodologies for the new kind of data market researchers have access to, in particular behavioural data. He is also a lecturer in Philosophy at Université Paris Ouest Nanterre, doing research on logic, philosophy of science and philosophy of statistics. Denis Bonnay was also a founder and director of data science at House of Common Knowledge. A former student at Ecole Normale Supérieure (Paris), he has a MSc in Logic & Foundations of Computer of Science (Université Paris VII) and a PhD in Philosophy of Science (University Paris I).
Maximum number of participants: No specific maximum
Will participants need to bring their own devices in order to be able to access the Internet? Will they need to bring anything else to the workshop? No

5:00 - 7:30DGOF Members Meeting
Session Chair: Otto Hellwig, respondi AG & DGOF, Germany
The DGOF members meeting is in German.
Room 248 
7:30 - 10:30GOR 19 Get-Together

Location: Weinladen, Im Ferkulum 30, 50678 Cologne

The GOR 19 Get-Together is open to anyone with a valid GOR 19 conference or workshop ticket! No tickets at the door!
Date: Thursday, 07/Mar/2019
8:00 - 9:00Begin Check-In
9:00Track A: Internet Surveys, Mobile Web, and Online Research Methodology I

Sponsored by aproxima
Room Z28 
9:00Track A: Internet Surveys, Mobile Web, and Online Research Methodology II

Sponsored by aproxima
Room 154 
9:00Track B: Big Data and Data Science

In cooperation with the International Program in Survey and Data Science (IPSDS)
Room 158 
9:00Track C: Politics and Communication
Room 149 
9:00Track D: Applied Online Research (Angewandte Online-Forschung)

In cooperation with
Room 248 
9:00 - 10:15Opening & Keynote
Room 69 

The Future of Consumer Insight in the Digital Era

Stefan Oglesby

data IQ AG, Switzerland

The phenomenon of “Big Data” is undisputed. More and more aspects of our day to day life will leave a digital trace. In parallel, new techniques that allow to store and analyze increasingly large amounts of data are emerging. However, pundits project different, sometimes contradicting views on the future of the consumer insights business. Some predict an accelerating substitution of surveys by Big Data in combination with powerful analytical tools. Others express a critical view on the quality and validity of Big Data and see a revival of “classic”, survey-based approaches.

The current practice of consumer research shows a pragmatic approach, aiming at an integrated view on various data sources. Amongst others, relevant challenges must be addressed:

  1. Consensus is needed on a scientifically founded new research paradigm for consumer insight.
  2. Solutions are required that ensure transparency with regard to the origin and quality of all data sources used to create insight.
  3. Specific ethics need to be developed with regard to data collection and, in particular, data linking as the probably most important future approach to relevant consumer insight.

Successful case studies endorse the business value created by the new paradigm:

  • Combining mobile sensor data with transaction data has helped retail stores to significantly increase the conversion rate of customers.
  • Integrating survey-based typologies with Big Data patterns opens up new opportunities for effective content marketing for a multi-channel commerce.
  • Digitizing the customer journey on a consistent “ground truth” allows unprecedented insights into the decision process.

Building on these case studies, the presentation will examine the challenges of the consumer insight business in the digital era.

Dr. Stefan Oglesby, MBA IMD, is founder of data IQ AG, a company specializing in customer insights and data analytics services, and lecturer for consumer research at the University of Lucerne. In his role as member of the Presidium of the Swiss Association of Market Research, he is contributing to the association's transition into the digital era and advocates its new positioning as a representative of all insights specialists - also beyond classical survey research. He has more than 20 years of experience in marketing and social research, including roles as research director and CEO at a leading Swiss market research agency.

10:15 - 10:45Break
10:45 - 11:45A02: New Technologies and Human-like Interviewing
Session Chair: Oliver Tabino, Q Agentur für Forschung GmbH, Germany
Room Z28 

Adapting surveys to the modern world: comparing a researchmessenger design to a regular responsive design for online surveys

Vera Toepoel, Peter Lugtig, Marieke Haan, Bella Struminskaya, Anne Elevelt

Utrecht University, Netherlands, The

Relevance & Research Question: In recent years, surveys are being adapted to mobile devices. This results in mobile-friendly designs, where surveys are responsive to the device being used. How mobile-friendly a survey is, depends largely on the design of the survey software (e.g. how to deal with grid questions, paging or scrolling designs, visibility, tile design etc.) and the length of the survey. An innovative way to administer questions is via a researchmessenger, a whatsapp-like survey software that communicates as one does via whatsapp (see In this study we compare a researchmessenger layout to a responsive survey layout in order to investigate if the researchmessenger provides similar results to a responsive survey layout and if the researchmessenger results in more respondent involvement and satisfaction.

Methods & Data: The experiment has been carried out in 2018 using panel members from Amazon Mechanical Turk in the United States. Respondents were randomly assigned to the researchmessenger survey or the regular responsive survey. In addition, we randomly varied the type of questions (long answer scale, short answer scale, open-ended). We used four blocks of questions containing questions about politics, news, sports, and health. To investigate question order effects-and possible respondent fatigue dependent on the type of survey- we randomly ordered blocks of questions. 1728 respondents completed the survey.

Results: We are currently analyzing results. We will investigate response quality (e.g. response distribution/mean scores, #check-all-that-apply, #don’t know, item missings and drop out, use of back button), survey duration, and respondents’ evaluation of the questionnaire. Respondents could self-select into a particular device. We will also compare results obtained via different device. We will show a video of the layout of both the researchmessenger and regular survey.

Added Value: The experiment identifies recommendable design characteristics for an online survey in a time were survey practitioners need to rethink the design of their survey since more and more surveys are being completed on mobile phones and response rates are declining.

Toepoel-Adapting surveys to the modern world-183.pdf

Voice Recording in Mobile Web Surveys - Evidence From an Experiment on Open-Ended Responses to the "Final Comment"

Konstantin Leonardo Gavras

University of Mannheim, Germany

Relevance & Research Question: In times of increased usage of mobile devices, the user experience has changed dramatically. Interacting with IoT using voice prompts has become common to wide shares of the public. However, survey research has not yet acknowledged these changes in online mobile behavior (Singer/Couper 2017). Most mobile web surveys only allow respondents to answer open-ended questions by writing them down manually. In order to realize the full potential of mobile devices, mobile web surveys should allow respondents to record their answers vocally. Using an experiment on mobile devices, I show that voice recording has potential for recruiting new respondents to open-ended questions, but slightly alters respondents’ behavior.

Methods & Data: The experiment was part of the GLES 2018 pre-test with 1566 respondents on mobile devices in Germany. Respondents were forced to take the survey with mobile devices to avoid self-selection. In order to avoid ceiling effects, I decided to employ the experiment with the final comment of the survey, forcing respondents to either comment manually or via voice recording.

Results: The results of this experiment provide evidence that using voice recording techniques allows survey researchers to recruit new target groups for open-ended questions in mobile web surveys. However, this innovation is accompanied by minor behavioral differences in response styles. Respondents who are older, have lower level of education and are less politically interested are more likely to use voice recording over being forced to write down open-ended responses. Furthermore, I am able to show that vocally recorded responses are on average friendlier than written comments, providing first evidence that social desirability bias might increase using this survey mode.

Added Value: Using a large-scale experiment, I was able to show that voice recording is a feasible alternative for gathering responses to open-ended questions in mobile web surveys. Besides increasing coverage in general, voice recordings are able to motivate underrepresented respondents in open-ended questions to provide answers. Using automated transcribing tools, voice recording allows researchers to ask additional open-ended questions in mobile web surveys, realizing the full potential of mobile devices for survey research.

Gavras-Voice Recording in Mobile Web Surveys-217.pdf

How well is remote webcam eye tracking working? - An empirical validation of Sticky and Eyes Decide against Tobii

Michael Wörmann

Facit Digital GmbH, Germany

Relevance & Research Question:

Keywords: Evaluation webcam eye tracking; Sticky, Eyes Decide, Tobii

Webcam remote eye tracking has been on the market for some time now, but is still viewed critically by many, as the accuracy of results measured by a webcam appears questionable. We evaluated two webcam eye tracking solutions, Sticky and Eyes Decide and compared them to Tobii – an established offline eye tracking solution. In addition to the comparison of the eye tracking results we also compared the handling and applicability of the two tools.

Methods & Data:

Keywords: UX laboratory setting; standardized conditions; specification of scenarios; moderated test; HD webcam; Tobii T-2-60;

30 individual interviews in the Facit Digital UX lab in Munich. 3 independent samples of 10 participants were tested either with Sticky, Eyes Decide or Tobii. Two German websites (Fressnapf / Capri Sun) were presented with suitable use cases. Heatmaps and selected areas of interests were compared. Metrics were compared using t-tests.


Keywords: Limited field of application; low fit of heatmap data; good fit of areas of interests

The heatmaps showed no good fit for Eyes Decide and Sticky with our reference Tobii. Also, the self-reported perceived areas of the participants only matched partly with the recorded heatmap data. Numeric values match somewhat better with Tobii for both tools.

Both, Sticky and Eyes Decide have constraints which limit their field of application: For Sticky, the high quota of not usable results and the missing raw data for individual participants only allow limited results and increase the recruiting effort. Eyes Decide only offers English participant instructions which limits the possible target group and has a tedious test setup.

Added Value:

Keywords: empirical analysis, independent, classification, areas of application

The study offers an empirical analysis of the advantages and disadvantages of two online eye tracking solutions in terms of test creation, conduction and results in comparison to Tobii. The study also allows a classification of the tools and shows suitable areas of application.

Wörmann-How well is remote webcam eye tracking working-113.pptx
10:45 - 11:45A12: Understanding Consumer Behaviour
Session Chair: Lisa Dust, Facts and Stories GmbH, Germany
Room 154 

In search of inspiration – Exploring the product category

Sophie Vogt

KERNWERT GmbH, Germany

Relevance & Research Question:

The increasing differentiation of target groups and the increased pace of product launches pose new challenges for the creatives in the communications agencies. Quantitative tests, which are often used for pitch preparations, help to gather basic market information, but there is often a lack of everyday input that brings the category and the target audiences to life - in short, non-verbal, emotional, associative input for the creation. The goal of our qualitative approach is to gain additional, inspiring insights on a specific category with the help of a short, efficient qualitative digital study in order to identify points of entry for communication and advertising initiatives.

Methods & Data:

Our qualitative "spotlight test" is a small, thematically focused and swiftly conducted study which illuminates a product category via digital diaries and creative activities.

In order to test the suitability for different topics the study will be conducted for two product categories: Smoothies and E-Scooter-Sharing. Each product category will be elaborated with 12 users and 12 non-users for 7 days. The online diary is self-documentation in text, image and video of the touchpoints with the product category (e.g. purchase, usage, preparation). Within the activities we combine associative and projective tasks with specific questions about the product group and the purchasing behavior (e.g. first contact, experiences, perception of brands, advertisement, packaging). The mix of ethnographic data and individual, uninfluenced answers allows to collect in a relatively short time holistic, diverse results that help to explore the category.


None yet. We will conduct an example study for our approach for two product categories (Smoothies and E-Scooter-Sharing), in January-February 2019.

Added Value:

This study aims to show how to explore and understand specific issues, behavior and feelings by means of a rapid and agile research design. It should also exemplify the broad fields of application of qualitative digital research, the study aims to show that qualitative digital research can be a valuable addition to traditional (quantitative) methodologies, e.g. when it comes to understanding a product category and to collect inspirational insights from the participant’s life.

Vogt-In search of inspiration – Exploring the product category-234.pdf

Believing in social proof or personal experience? - Contrasting and comparing the effect of different kinds of eWOM in online shops

Christian Bosau, Levi Meyer

Rheinische Fachhochschule Köln, Germany


In the online-marketing literature the question regarding electronic word of mouth (eWOM) is not clarified yet whether total ratings (i.e. the total mean of many users’ quantitative rating) in online shops influences the perception of customers more than a personal rating (a single subjective text-based review). Many arguments expect that the first one – representing social proof – is more powerful than the second one – representing direct personal experiences –, compared in this study for well-known branded products and their no-name counterparts.


This 2x2x2 experimental online-study (N=166, non-probability) modified indicators for social proof (total evaluation: positive vs. negative, repeated measurement) as well as direct experience (two personal reviews: positive vs. negative each, repeated measurement) for different products (branded vs. no-name mobile speakers), testing their main and interaction effect on three indicators of customer perception (product quality, buying intention, recommendation to friends). Hence, subjects (randomized assigned to branded or no-name products) assessed in total 4 products each (randomized presentation).


Not surprisingly, the implementation of the experimental design created two significant main effects: a positive total evaluation (p < .00, part. Eta2 = .49) and positive personal reviews (p < .00, part. Eta2 = .53) improved the overall perception. More interestingly – against the original hypothesis – the effect of direct experiences was even a bit higher than the influence of social proof.

As expected, besides the clear main effect of the brand factor (branded products are higher rated than no-name-products; p < .05, part. Eta2 = .03), a significant interaction effect showed (p < .05, part. Eta2 = .03) that a buffering effect can be found for branded products: they are – in direct comparison – better rated than their no-name-counterparts, if both ratings (total and personal) are negative.

Added Value:

Compared to many other simple studies before this experiment manipulated simultaneously personal ratings and the overall total evaluation and therefore was able to compare the strength of the effect sizes directly. The study shows that even a few negative personal ratings can have a strong negative effect on customers’ perception even if the overall rating is still high.

Bosau-Believing in social proof or personal experience-129.pdf

Recreational gaming – dependence and social problems as outdated concepts in a new world of gaming?

Birgit Ursula Stetina, Jan Aden, Anastasiya Bunina, Carolin Griehsler, Zuzana Kovacovsky, Reinhard Ohnutek, Armin Klaps

Sigmund Freud University, Austria

Relevance & Research Question: Discussions about problematic aspects of online gaming and the “Gaming Disorder” are again present in daily press and professional discussions because of recent changes in ICD-11. Although many experts agree that clinicians should be careful with the diagnosis the public opinion seems different. However, earlier studies already showed that only a very limited number of gamers fulfill the criteria for gaming disorder or other forms of Internet dependency with gaming genres playing a relevant role (eg Stetina et al. 2011).

Objective of the presented study was to evaluate a gender balanced sample according to their gaming routine and clinical aspects as dependency and anxiety.

Methods & Data: Using an online questionnaire 147 gamers were surveyed (female:n=66, male:n=81) in a cross-sectional design with several (clinical) scales such as IGD-20 (eg Pontes et al. 2014), SIAS (Matttick & Clarke, 1989) and SPIN (Connor et al., 2000).

Results: First of all results show that the sample includes no dependent gamer (cut-off 71). But the results show a significant difference between males and females with female gamers (M=33.33,SD=11.28) showing significantly less symptoms (T(145)=-2.561,p=.011) than men (M=38.06,SD=11.01); both groups showing no clinically relevant signs of Internet Gaming Disorder. No gender differences were found in the sum scores of the instruments measuring social anxiety (SIAS:(T(145)=-0.39,p=.694, SPIN:(T(145)=1.18,p=.239). Only 8.2% (n=12) participants, similar to the general public, show clinically relevant scores using a cut-off of 30. This percentage is slightly higher using the SPIN scores with 81.6% (n=120) of inconspicuous participants (cut-off 19). However the SPIN category “mild social phobia” has to be considers (11.6%, n=17) and therefore the instrument shows quite similar results.

Added Value: Recreational gaming is often discussed as problematic behavior, although educational and therapeutic games are on the rise. Pathologizing is not the answer. It seems that more than ever it is highly relevant to think of gaming as normal and average behavior, independent from the purpose of the game. Independent from the well-known differences between genres we should start thinking about gaming as a potential adaptive coping strategy and part of our daily lives (eg casual games).

Stetina-Recreational gaming – dependence and social problems as outdated concepts-199.pdf
10:45 - 11:45B02: Text Mining and NLP
Session Chair: Florian Keusch, Universität Mannheim & DGOF, Germany
Room 158 

Towards the Human-Machine-Symbiosis: Artificial Intelligence as a Support for Natural Language Clustering

Marc Egger, André Lang

Insius, Germany

Over the past years, the amount of available natural language speech data has risen tremendously. People use Social Media Channels to express their opinions in natural language text or interact with digital assistants like Amazon Alexa or Apple Siri via their voice. Most recently, market researchers have also proposed novel survey designs, where participants can provide answers via natural speech interaction. This opens up a novel and vast data source to draw rich insights from the unfiltered voice of consumers/participants. However, due to the amount of available natural language data, the velocity of its creation, next to the requirements on the insights’ depth, novel analysis methodologies are required. On the one hand, “human-powered” qualitative analysis methods offer deep insights but lead to tremendous manual efforts and thus are hardly applicable in big data scenarios. On the other hand, automated “machine-powered” methods (e.g. using natural language processing) can be applied in big data scenarios but only offer shallow insights due to the complexity of human language.

Considering Lickliders` (1960) vision on human-machine-symbiosis, we claim to the construction of novel analysis methodologies where human and machine work hand-in-hand to perform better in cooperation than individually. The research at hand therefore proposes a novel human-machine cooperative methodology to derive the most important topics from large text collections by semi-automated clustering of natural language concepts.

We apply our methodology on a data set of ~20.000 texts and describe a software artifact to illustrate the research process. The artifact implements the method and allows researchers to cluster automatically elicited concepts and phrases to more abstract topics. Our methodology proposes an additional “relevance-feedback loop” that utilizes an artificial neural net for suggesting concepts that might be clustered next. As an evaluation of our methodology, we compare the results of human-machine clustered concepts to those that were elicited automatically.

Our initial results show that the cooperation of machine and human clearly leads to more rapid insights than manual qualitative approaches, while also offering deeper insights than purely automated approaches. Furthermore, our research reveals that a linear process and a “definition-of-done” is necessary for human-machine-cooperative scenarios.

Impact evaluation by using text mining and sentiment analysis

Cathleen M. Stuetzer, Marcel Jablonka, Stephanie Gaaw

TU Dresden, Germany

Relevance & Research Question: Web surveys in higher education are particularly important for assessing the quality of academic teaching and learning. Traditionally, mainly quantitative data is used for quality assessment. Increasingly, questions are being raised about the impact of attitudes of the individuals involved. Therefore, especially the analysis of open-ended text responses in web surveys offers the potential for impact evaluation. Despite the fact that qualitative text mining and sentiment analysis are being introduced in other research areas, these instruments are still slowly gaining access to evaluation research. On the one hand, there is a lack of methodological expertise to deal with large numbers of text responses (e.g. via semantic analysis, linguistically supported coding, etc.). On the other hand, deficiencies in interdisciplinary expertise are identified in order to be able to contextualize the results. The following contribution aims to address these issues.

Methods & Data: An annual online survey of lecturers regarding the quality of academic teaching and learning was conducted within a selected university in Germany between 2013/14 and 2017/18. Information regarding the open-ended question of what is particularly important in the teaching process were extracted by using text mining methods and evaluated by using sentiment analysis.

Results: The results of the analysis of the text data of 791 respondents (lecturers) show their different attitudes towards the quality of teaching. This will be merged with results of the annual quantitative online survey of students (n=6.615, between 2013/14 and 2018/19) regarding the question what is actually conveyed in teaching processes. Comprehensive results are work in progress.

Added Value: The presentation will show how this case study contributes to the field of impact evaluation and reveals methodological implications for the development of text mining and sentiment analysis in evaluation processes.

Stuetzer-Impact evaluation by using text mining and sentiment analysis-221.pdf

What to expect from open-ends?

Eva Wittmann, Sara Wilkinson, Cecile Carre


Relevance and Research Question:

Open-ends have been a constant in survey-based research, as they add qualitative insights and enrich quantitative data by encouraging self-expression for respondents. But research in recent years has fundamentally changed: it is getting more global, technology is getting increasingly diverse, and increasingly embedded in our daily lives, and consumers are becoming less engaged by the idea of participating in research projects.

Yet researchers and clients still expect to gain insights via traditional open-ends. This presentation aims to examine where we currently stand with open-ends and whether technology and structural settings are changing this outcome.

Methods & Data:

To review the status quo on open-ends, we have analyzed 40 Projects from 4 countries, approx. 15K open-ends. For additional insights on how respondents use new technology (e.g. audio or video), we ran an adhoc project giving respondents from Ipsos Panels the option to use these new technologies to replace traditional open-ends.


Our analysis shows how that differences in open-end quality can be linked to specific factors such as culture, gender and age or even device used. Those factors accounted for differences between 10%-40% in response quality. Opening open-ends to new technologies (such as video or audio) suggests that these quality indicators can be improved, if we can get audiences to more widely embrace these new tools.

Added Value:

This presentation will give a holistic summary of the current situation of open-ends in online research and an empirically-based outlook on how technology can change the future of this type of questions.

Wittmann-What to expect from open-ends-152.pptx
10:45 - 11:45C02: Fake News, Fake Users
Session Chair: Simon Munzert, Hertie School of Governance, Germany
Room 149 

Integrating Artificial intelligence (AI) and the Human Crowd to Tackle 'Fake News': A Design Proposal

Claudia Loebbecke1, Tim Schumacher2

1University of Cologne, Dept. of Media and Technology Mgmt., Cologne, Germany; 2EYEO GmbH, Cologne, Germany

Relevance & Research Question:

Fake news has been around since politics began. Information - especially news - is often construed with 'truth' simply because online content seems to reflect the 'real world'. However, our conscious or unconscious agreement or disagreement with an author's words does not make them true or untrue. Key to successfully detecting fake news is reliable and unbiased data sources. They are exceedingly hard to get or openly accessible. The question arises how we can deploy technology to inform users of the trustworthiness of a particular piece of news or source of information.

Methods & Data (Proposed Design):

We conceptualize AI-based decision 'support' with human content rating (crowdsourcing paradigm) to tackle the challenge. In a first design phase, an open-source browser extension provides access to Metacert's data sources of fact-checking websites and uses that data to rate the trustworthiness of (inspected) news websites. A simple traffic light system - based on straight-forward (transparent) analytics - points to the trustworthiness of web content or entire websites leaving it up to the user to continue reading. Users can anonymously offer feedback likely leading to an increasing number of instantiations and degree of granularity. After evaluation by fact checkers, the accumulated feedback shall be used to continuously update the database underpinning the system.

In a second design phase, this 'crowd collected' data continuously feeds the AI-grounded sensing for supervised learning. While early AI-based ratings are likely to be wrong, the designed system should 'learn' fast from many user inputs. Eventually, moving ratings to the blockchain to improve transparency and third-party contributions to the database.

Results & Added Value:

The design detects 'fake news' with strong network effects (value of a good to one user depends on the number of other users) on a multi-sided platform (users, data providers, and web publishers) requiring trust resulting from credible input data among data providers, web publishers, and users.

Fact or Fake? A mediapsychological perspective on children judging credibility of news

Astrid Carolus, Münch Catharina, Sammueller Raphaela, Schwing Michelle

Universität Würzburg, Germany

Relevance & Research Question

The internet and particularly social media have become an important source of information resulting in an increased risk to encounter so-called “fake news”. Consequently, information competence - the ability to detect incorrect and unreliable information - constitutes a key subdomain of digital competence. From a mediapsychological perspective, this study aims for (1) a methodological approach to assess information competence and (2) initial insights into children’s information competence.

Methods & Data

In a pretest (n = 53 students), four news articles (two “true”, two “fake”) were selected as stimulus material. A total of 247 German students (127 male) participated in the main study, ranging in age from 10 to 19 years (M = 13.66; SD = 2.36). In a 2x3x3 experimental design, participants were randomly assigned to two news articles (“true” vs. “fake”) with articles varying regarding channel (print, online, social media), source (Bild, Süddeutsche, anonymous) and topic. Participants evaluated these items regarding credibility and verisimilitude. Further, self-reports asked for impulsivity, extraversion, need for cognition (nfc), media use and information competence (e.g. trust in media, preferred sources of information).


Descriptive analyses revealed 83.62 % of the “true articles” and 77.86 % of the “fake articles” to be identified correctly with print articles to be perceived as most truthful. Interindividual differences affected the evaluation of verisimilitude. Hence, detecting “fake article” incorrectly was significantly negatively associated with age (r = -.145, p = .001). Further, nfc and personality were significantly correlated with different aspects of children’s media use. The final model with all predictors (nfc, extraversion, impulsivity, media use, source, channel, topic and age) accounted for a significant proportion of the total variation in participants’ evaluation of verisimilitude (adj.R2 = .424, F[41, 452] = 9.849, p < .001) with nfc, age, topic and newspaper use to contribute significantly.

Added Value

Information competence has become essential in our digitized news world. Our study reveals first insights regarding children’s evaluation of news articles and their ability to distinguish correct from incorrect information. First conclusions about constituting predictors of information competence are drawn and first ideas of pedagogical interventions are derived and discussed.

Fake it till they take it? Pseudo user effects and pseudo user literacy

Niels Göran Mede

Universität Zürich

Relevance & Research Question: Social bots, click farm employees, and micro workers—so-called pseudo users—can inflate the number of likes of online messages. Thus, they manipulate genuine users’ credibility perceptions of, attitudes towards, and intentions to engage with (political) online messages. One way to fight this effect is by fostering pseudo user literacy. However, experimental evidence for this effect is missing. The present study tackles this gap by exploring the following questions: How does endorsement by large amounts of pseudo users affect 1) credibility perceptions of, 2) attitudes towards, and 3) intentions to engage with social media content among pseudo-user-literate and pseudo-user-illiterate individuals?

Methods & Data: I conducted an online survey with a 2 (information about pseudo users vs. control) × 2 (pseudo user likes vs. control) between-subjects design (N = 201). To increase variance in pseudo user literacy throughout the sample, participants were randomly assigned to watch a video about social bots before completing a literacy measure. Subsequently, an Instagram post by a fictitious health insurance company was shown to all participants. I chose this topic because pre-examinations suggested that respondents would not collectively tend towards a pro or a contra position. Participants saw one of two versions of the post. One had zero likes, the other one was liked by 316.609 pseudo user accounts, as respondents were forced to find out.

Results: Three moderation analyses tested the research questions (predictor: pseudo likes; moderator: literacy; dependent variables: credibility perceptions, attitudes, or engagement intention). Neither significant unconditional nor conditional effects were discovered in any of the analyses.

Added value: The analyses indicate that even those social media users who are not aware of pseudo users may not be particularly prone to have their credibility evaluations, attitudes, and behavioral intentions sabotaged by them. From that perspective, pseudo users may not be a threat to online discourses as dystopian debates have suggested. Meanwhile, the results also suggest that literate individuals do not make very great efforts to defend their evaluations, attitudes, and behaviors against manipulation attempts of pseudo users. This finding underlines the need for critically evaluating the uniform call for media literacy campaigns.

Mede-Fake it till they take it Pseudo user effects and pseudo user literacy-176.pptx
10:45 - 11:45D02: GOR Best Practice Award 2019 Competition I
Session Chair: Alexandra Wachenfeld-Schell, GIM Gesellschaft für Innovative Marktforschung mbH & DGOF, Germany
Session Chair: Otto Hellwig, respondi AG & DGOF, Germany
Room 248 

The Innovation Research Game Changer: tuning research to Henkel’s agile fuzzy front end of innovation

Anita Peerdeman-Janssen1, Vera Diel2

1InSites DE GmbH; 2Henkel AG, Germany

Early 2018, Henkel was challenged to find a disruptive way to tune market research to the needs of their Game Changer Innovation team, a team with the challenging mission to come up with new concepts fast, to match the fast-paced reality. In order to support this, the Henkel Consumer Insights team was looking for research solutions that embraced agility, iterative testing and fast cycle times, whilst being consumer-centric and fully hybrid.

To serve the various research needs in their innovation cycle, an online network gathering 1,500 USA participants was set up. This network connected both NYC residents (n=500) and consumers reflecting the general population of the US.

Using the proprietary Square® technology, hybrid studies were conducted in multiple innovation domains. Qualitative challenges were launched among the NYC sample, which mainly consisted of innovators and early adaptors, while the quantitative challenges, focusing on screening and validation, were launched among all Square members. All studies were customized to Henkel’s needs and then scaled up to deliver the required speed.

This new approach to innovation research proved to serve the need for speed: in just four months, 24 different studies were conducted. Moreover, the iterative and fast testing resulted in over ten high-quality concepts, ready to enter the next stage in the innovation funnel. Lastly, this new approach led to a high satisfaction amongst the Henkel teams, with those involved evaluating it with a score 9 (out of 10).

In this fast-paced reality, many organizations are challenged to move towards a more agile approach to innovation. Yet research is often not tuned to this new reality. The installment of network-based research to structurally collaborate through iterative research tasks has proven to entail many opportunities. Apart from being tuned to Henkel’s new, agile way of working, this new online research approach also increased consumer centricity at Henkel. Iterative testing forces the integration of consumer feedback in every step of the process, resulting in better-quality consumer insights and concepts. We believe this case study could inspire many researchers and marketers who experience this common challenge.

The Online Overload: Predicting Consumer Choice in a Digital World

Julia Görnandt1, Sander Noorman1, Kris Compiet2

1SKIM, Germany; 2Vodafone Ziggo, The Netherlands

Today’s technology is disrupting consumer expectations, how they shop and interact with brands. Consumers are more empowered and have more choices. In Telecoms, the shift to digital is already long underway: Online channels are key for the consumer acquisition and retention game, with consumers purchasing their subscriptions on either providers’ or comparison websites. To understand and predict consumer choice for telecom subscriptions, traditional conjoint is a widely used technique. However, it is crucial to replicate consumer choice in the most accurate way to understand and predict market reality as good as possible. Traditional approaches are stretched to their limits when it comes to mimicking online decision environments or in markets where the number of product choices is high.

In order to address the above-mentioned challenges in the context of telecom subscriptions, SKIM has replicated an online comparison website for a recent conjoint study in collaboration with VodafoneZiggo. Respondents repeatedly chose their preferred telecom subscriptions, from a large set of subscriptions offered by various brands. They could use filters and ranking variables, just as they would be able to do on an actual comparison website. On the back-end, a conjoint design was used to vary the subscriptions shown in each task. At the same time, a more traditional conjoint survey was run, allowing to compare results between the 2 approaches.

A comparison of both approaches proves that the online comparison website exercise is bringing us one step closer to reality and allows us to more accurately predict consumer choice. As hypothesized, preferences of respondents are more in line with actual market shares and switching rates between brands are much more realistic. Furthermore, the usage of filters and sort functions in the exercise gives us insight into how consumers simplify their choice process in a world of infinite options.

This new methodology allows for a more accurate understanding and prediction of consumer choice in an online world. In consequence, companies can make better decisions what products and prices to offer to help consumers choose their products over the competitors’.

Görnandt-The Online Overload-254.pdf

How to allocate resources best – case study of a nationwide newspaper

Annika Gröne1, Patricia Kehm1, Mario Lauer2

1DCORE GmbH, Germany; 2Süddeutsche Zeitung GmbH

Relevance & Research Question: Süddeutsche Zeitung (SZ) was planning on optimising their spending on the regional parts of their print edition for 2018. Thus, specific parts of the content needed to be analysed with regard to its relevance for subscribers. Therefore, this study was aimed to measure how extensively certain parts of the newspaper are read, if they meet subscribers’ expectations and how important they are to customers when it comes to the decision to keep or cancel the SZ subscription. Consequently, the focus of the study was to develop a measurement that shows the probability of terminating the subscription if a certain part will be removed from the newspaper.

Methods & Data: To answer these questions an online survey was conducted among subscribers. To ensure a reliable recognition and assessment of the several newspaper parts, one newspaper edition was displayed on screen by using the LASSO Software, a special method to measure reading behaviour within an online-survey, developed by DCORE. This software enables scrolling realistically through the newspaper and thus, measuring the key performance variables with high validity. To quantify satisfaction for each part, the “KANO model” was used. According to this, each part of the content has been defined with respect to its relevance / dispensability based on users’ evaluations. To take into account all relevant variables, the so called “Reflection-Score” for each part of the newspaper was calculated. This score provides a basis to predict the percentage of subscribers who could eventually terminate their contract, if a certain part of content is removed from the print edition.

Results: Based on the Reflection-Score the analysis showed clearly which parts of the SZ appeared to be most relevant and thus, were recommended to be kept as they seemed substantial to customer satisfaction and which parts were irrelevant with respect to the continuation of the subscription.

Added Value: The developed measurement of the study brought valuable insights to SZ on how to optimise the allocation of their resources to the respective newspaper parts and content. Additionally, the study can make a contribution on how subscription marketing can be optimised.

Gröne-How to allocate resources best – case study of a nationwide newspaper-237.pdf
11:45 - 12:00Break
12:00 - 1:00A03: Recruitment of Respondents and Participants
Session Chair: Jan Karem Höhne, University Mannheim, Germany
Room Z28 

Using Cash Bonuses for Early Participation to Improve Postal Recruitment of a Probability-Based Online Panel

Barbara Felderer, Ulrich Krieger

SFB 884, University Mannheim

Relevance & Research Question

Past research has shown that cash incentives are effective to increase response rates and recruitment in panel studies. However, there are gaps in the literature when it comes to the use of bonus incentives for early participation to facilitate the recruitment effort.

Our aim is to use conditional cash incentives for early participation to encourage respondents to sign up early during the field period, hence reducing the subsequent reminder effort. In addition, a smaller sample is needed to reach the recruitment goal.

We test the effectiveness of bonus incentives in increasing sample sizes and reducing fieldwork cost. In addition, we test for possible negative effects of the incentive treatment such as sample bias or early panel drop out.

Methods & Data

To test the effects of bonus incentives we implemented a large-scale experiment in the 2018 recruitment of the probability based German Internet Panel (GIP). The recruitment was based on a population register and conducted via postal mail.

For the experiment, 4800 sample cases were randomly assigned to three treatment goups: 1200 sample cases recieved a 50€ bonus incentive for early registration, 1200 a 20€ early registration bonus, and 2400 were assigned to the control goup that did not receive any bonus incentive. All sample members received a postal mail invitation including login information for the online recruitment survey including a 5€ unconditional cash incentive.


First analyses show, that the response rate is higher for the bonus groups than the control group but does not differ between the two bonus groups. No effect of the treatment on the sample composition of gender, age and german-non german citizenship can be found.

Added Value

Our study informs about the short and long term benefits and risks of bonus incentives on early registration for recruting respondents.

Text Message Invitations as a new way to conduct population wide online surveys? – Biases and Coverage Issues

Christoph Beuthner, Matthias Sand, Henning Silber

GESIS Leibniz Institute for the Social Sciences, Germany

Relevance & Research Question:

Online surveys are the most popular survey mode today. Besides all their advantages, sampling can be a downside of online surveys. Especially population-wide probability sampling is a great challenge for most web-based surveys. Therefore, we explore the usage of a sample consisting of randomly generated numbers, to recruit respondents via text message. Our goal is to assess if this method is feasible to conduct population-wide online surveys and by which biases such a survey might be influenced.

Methods & Data:

We used modified RDD sampling to create a sample of privately used mobile telephone numbers and then used an HLR Lookup procedure to remove invalid numbers. Afterward, we contacted the remaining numbers via SMS, which includes a short text about the length of the questionnaire, the name of our institution and a link to the survey. Respondents were randomly assigned to one of three questionnaires that differed in length (5, 10 or 20 Minutes). All questionnaires contained a core part, concerning demographics, selected attitudinal questions, questions concerning the political behavior, and questions regarding data linkage. The data was collected in November 2018.

In our analysis, we will investigate biases in the sample and compare our results to population benchmarks such as the turnout of the last election. Furthermore, we will compare the results of the data linkage part, to results data we obtained with an online access panel. Additionally, we will analyze the questionnaire length experiment.


The survey is in the field at the moment. We send a total of 11820 Messages. Thus far, the response rate is relatively low.

Added Value:

Our study will help to evaluate a new sampling method for web surveys. Analyzing biases created by the sampling method can help to identify for which purposes our procedure is feasible and how data quality might be affected. Furthermore, our experimental design will help to give recommendations regarding questionnaire length that is most suitable for an online survey based on SMS sampling.

Beuthner-Text Message Invitations as a new way to conduct population wide online surveys – Biases and C.pptx

Participant Recruitment Methods can Affect Research Outcomes: Personality Biases in Different Types of 'Online Sample'.

Tom Buchanan

University of Westminster, United Kingdom

Relevance & Research Question: Samples for online research are recruited in a number of ways (e.g. online panels, volunteer requests, crowdsourced labour marketplaces such as MTurk). Panel providers and researchers correctly pay attention to data quality and demographic representativeness of participants. However, biases in sample makeup with respect to motivation or personality are seldom considered. Could participants recruited in different ways produce different findings? For example, Openness to Experience is known to affect political voting preference. Might a sample skewed towards high Openness behave differently to one with a bias towards low Openness in research on political preference?

Methods & Data: In Study 1, a pseudo-experimental design compared personality scores of students participating for course credit, with those of individuals completing an online personality questionnaire on a voluntary basis. In Study 2, a correlational design explored whether personality affected political voting preference. In Study 3, personality scores of volunteers were compared with those of members of a commercially sourced, paid, online research panel. The potential effect on voting preference was evaluated.

Results: Results generally indicated that personality profiles were influenced by recruitment method. Volunteers had lower Extraversion, lower Agreeableness, lower Conscientiousness, higher Neuroticism, and higher Openness to Experience than people participating as a class requirement. Study 2 showed that Openness differences of the magnitude seen in Study 1 could affect voting preference. Study 3 demonstrated that findings obtained with volunteer participants were not replicated with paid panel members.

Added Value: This project extends existing work on online data quality, showing that personality biases may arise from recruitment methods. It demonstrates that these differences, while small, could affect research outcomes in meaningful ways.

Buchanan-Participant Recruitment Methods can Affect Research Outcomes-154.pptx
12:00 - 1:00A13: Data Quality in (Mobile)Web Surveys
Session Chair: Olga Maslovskaya, University of Southampton, United Kingdom
Room 154 

Out of sight, Out of mind? Survey Modes Effect in objective and subjective questions

Joachim Schork2, Cesare Antonio Fabio Riillo1, Johann Neumayr2

1STATEC research, Luxembourg; 2STATEC, Luxembourg

Web questionnaires are increasingly used to complement traditional data collection in mixed mode surveys. The flexibility of mixed modes provides many advantages such as less nonresponse issues, lowered expenditures, and compensation for the decreasing availability of other data sources, i.e. fixed-line telephone numbers. However, the increased usage of web data raises concerns whether web questionnaires lead to mode-specific measurement bias, since responses given to web questionnaires may be significantly different compared to other survey modes.

We argue that the magnitude of measurement bias strongly depends on the content of a variable and investigate differences between web and telephone data in terms of objective and subjective variables. The study is based on the Luxembourgish Labour Force Survey that collects both objective and subjective employment variables. Analysis of the raw data reveals significant differences in sample composition (e.g. participants' personal characteristics such as age or nationality) as well as in the objective variable employment status and the subjective variables wage adequacy and job satisfaction.

In order to investigate whether differences in employment variables are caused by sample composition or mode-specific measurement bias, we match web and telephone samples according to variables that lead to dissimilarities in sample composition. We identify these variables by a combination of automatic variable selection via random forest and a theory driven selection. Based on the selected variables, we then apply a Coarsened Exact Matching that approximates randomized experiments by reducing dissimilarities between web and telephone samples.

After matching, we show that employment status is not affected by mode-specific measurement bias, but web participants report lower levels of wage adequacy and job satisfaction. Even though further research on subjective variables is advisable, our results support the implementation of mixed survey modes in official statistics such as the Labour Force Survey.

Attention checks in web surveys: The issue of false positives due to non-compliance

Henning Silber, Joss Roßmann, Tobias Gummer

GESIS - Leibniz Institut für Sozialwissenschaften, Germany

Relevance & Research Question:

All survey and especially web surveys rely on respondents being mindful when answering the survey questions. Survey researchers have, therefore, developed a variety of indicators to assess the data quality of survey data (e.g., straightlining, item nonresponse, and speeding). An indicator that has become especially popular in market research is attention check questions. Attention check questions usually explicitly instruct respondents to provide a specific answer or select a specific response category. However, these tests may produce false positives. That is, respondents fail the attention check tasks on purpose and are wrongfully deemed inattentive. These false positives endanger to confound the measure of attentiveness. Our study contributes to the knowledge about attention checks by reporting the findings of two experiments on how to optimize these checks.

Methods & Data:

Both experiments were conducted in a web survey based on a sample drawn from a German non-probability access panel (N = 3000). Respondents were randomly assigned to either answer on PC or smartphone. In the first experiment, we manipulated reasons that were given to respondents why they should comply. In the second experiment, we varied the placement of the attention check question within a grid question to assess the capability of these checks in measuring attention.


The results of the first experiment show that more respondents pass the attention check if a specific reason is given, which suggested that the measure might be confounded with compliance. These results are complemented by the second experiment, in which we found that respondents are more likely to pass the test if the check was placed earlier in the question sequence, thus, suggesting that attention checks—while having its inaccuracies—are capable of measuring attention.

Added Value:

The presentation provides new insights into the usefulness of attention check, a tool which is frequently used in web surveys to assess the mindfulness of respondents and data quality in general. In addition, we provide recommendations on how to design these checks to provide meaningful measures.

Effects of Survey Design and Smartphone Use on Response Quality: Evidence from a Web Survey Experiment

Joss Roßmann

GESIS Leibniz Institute for the Social Sciences, Germany

Relevance & Research Question

The increasing prevalence of respondents answering web questionnaires on their smartphone poses a challenge to researchers to optimize the design of their surveys for the use of specific devices or to implement designs that allow adaptation to the respective devices used. While applying non-adaptive designs might impair response quality on some of the devices, adaptive designs may limit the comparability of results between the devices used. Therefore, this study examined how four different adaptive and non-adaptive designs affected response quality on smartphones compared to other devices.

Methods & Data

Respondents from an opt-in online panel were randomly invited to participate in our survey experiment either on a smartphone or on a PC, Notebook, or Tablet. Then, the respondents from both conditions who complied with the instruction were randomly selected to answer the survey either in a non-adaptive PC-optimized, an adaptive, or in a non-adaptive smartphone-optimized design with paging- or scrolling layout. Overall, 4.299 respondents participated in the 2x4 fully factorial web survey experiment. We used regression modeling to study the effects of survey design and device on several indicators of response quality, such as interview duration, straightlining, non-substantive answers, and on the substantive response (i.e. latent means).


The results of our study showed that particularly answering the survey in a non-adaptive PC-optimized design on a smartphone impaired the respondents’ survey experience and increased the perceived and actual interview duration as well as the number of survey breakoffs. While the non-adaptive smartphone-optimized design with a paging layout produced longer interviews, we did not find negative effects of smartphone-optimization on response quality on either type of device. In addition, the effects of the survey design on the substantive response were mostly small and insignificant. However, our results also indicated that response quality may slightly differ between devices in an adaptive design.

Added Value

Our study shows that applying a non-adaptive PC-optimized design is the least good option for surveying samples which include non-ignorable numbers of smartphone respondents. Thus, we recommend implementing either adaptive or non-adaptive smartphone-optimized designs in order to achieve high response quality for mixed-device surveys.

Roßmann-Effects of Survey Design and Smartphone Use on Response Quality-153.pdf
12:00 - 1:00B03: Data from Video and Music Platforms
Session Chair: Simon Kühne, University Bielefeld, Germany
Room 158 

Why not to use popularity scores from platforms. The hidden biases of YouTube data

Merja Mahrt

Heinrich-Heine-Universität Düsseldorf, Germany

Relevance & Research Question: Many social media platforms give access to information such as number of likes, shares, or views of their content items, facilitating large-scale analyses of content popularity. These data, however, are likely to contain “hidden biases” (Crawford, 2013), partly because of intransparent algorithmic decisions that determine what is included in such data (Gillespie, 2012). Studying the popularity of YouTube content, how do popularity scores from the platform itself compare to other data sources?

Methods & Data: A content analysis of the daily YouTube top 10 in Germany examines popularity, genre, and author distribution (1,164 unique videos from March-August 2014 were collected). In an online survey in September 2014, 1,665 respondents were asked for their use of online video sites and awareness of then recent popular online videos. Clickstream data from Nielsen Germany for June 2014 were likewise analyzed for popularity of content (data on 8,147 YouTube users who watched around 250,000 different videos were acquired from Nielsen).

Results: The three data sets show very different sides of what was popular in spring and summer 2014 in Germany. The survey as well as the clickstream data reveal clear biases in YouTube use by age, gender, education, income, and apparent content interest. The most popular videos from YouTube’s own top 10 and Nielsen’s clickstream data only partly match, with gaming being dominant according to YouTube itself, while Nielsen users concentrated much more on then current music videos.

Added Value: The comparison of three data sources highlights that platform scores for popularity are essentially black boxes that may contain numerous biases due to over- or underrepresentation of users and intransparent platform decisions about what is included or even advertised to the larger usership. Research that solely relies on platform data thus risks systematic biases that put into question the validity of this approach to social research.


Crawford, K. (2013, April 1). The hidden biases in big data. Harvard Business Review Blog. Retrieved from

Gillespie, T. (2012). Can an algorithm be wrong? Limn, n.v.(2). Retrieved from

Mahrt-Why not to use popularity scores from platforms The hidden biases-177.pdf

Rank eater versus Muggle: The impact of the two consumer orientations on the ranking in the digital music market

Junmo Song, Eehyun Kim

Yonsei University, Korea, Republic of (South Korea)

Relevance & Research Question:

How do consumers behave in the digital music market? Previous discussions on the music market assume consumers who respond to reasonable factors such as the quality and cost of the song. In the real music market, however, consumers often show unconditional preference for certain singers or songs, and they are organized into groups called fandom, which also exerts social influence. Therefore, the aim of this study is to show the presence of ‘collective pick’ carried out by organized consumers and to demonstrate the impact of collective pick on the music market.

Methods & Data:

This research is based on hour-based ranking data of 1,898 songs from ‘Melon’, ‘Genie’, ‘Mnet’, ‘Bugsmusic’, which account for 70 percent of the total streaming market share in South Korea, from May 27 to December 31, 2018. Hourly data enables the operational definitions as follows: the ranking early in the morning is highly dependent on the repetitive streaming of organized consumers, and the ranking at the end of the day reflects the impact of the general consumers. In other words, the difference in rank between two time zones represents the boosting effect of the collective pick. In this study, we analyze the effect of collective pick on the short and long-term performance of a song through fixed effect panel analysis, sequence analysis, multinomial logistic regression.


Depending on the characteristics of a singer, song and platform, the consumer's collective pick works differently. The chart preemption effect of this fan-boosting allows the song to outperform at the daily life cycle level. On the long-term life cycle level, fan boosting has a positive effect on the long-term survival of the song in the chart. Boosting by fans have a significant effect in both short and long-term level even when controlling other factors such as the initial entry rank of music and the characteristics of singers.

Added value:

This study is meaningful in that it confirms that the performance of a song in the digital music market is not simply determined by the quality and cost of music, but by the collective pick of fans.

Song-Rank eater versus Muggle-205.pdf

Methods and Tools for the Automatic Sampling and Analysis of YouTube Comments

M. Rohangis Mohseni1, Johannes Breuer2, Julian Kohne2

1TU Ilmenau, Germany; 2GESIS – Leibniz Institute for the Social Sciences, Germany

Relevance & Demand: YouTube is currently the largest and most important video platform on the Internet. For young people, YouTube is already partly replacing television. While there exist a number of studies on the use of YouTube, there are comparatively few quantitative empirical studies that deal specifically with user comments. As suggested by Thelwall (2017), we believe that part of the reason for this is that researchers are not aware of the potential of the YouTube API or do not know how to use it for their research.

Methods/Tools: We compared different tools for sampling and analyzing YouTube comments, paying special attention to the functionality and usability of the options in order to arrive at a practical decision-making aid for various use cases. We investigated the possibilities of automated evaluation (sentiment analysis of text and emojis, content analysis, topic modelling), but also the problems and limitations associated with typical YouTube comments (e.g., use of irony, slang, unusual words, creative spelling). In this process, we especially focused on challenges in working with emojis (parsing; display differences between platforms; cultural and inter-individual differences in the use and interpretation of emojis, etc.).

Script Creation & Use: We created an R script (doi:10.17605/OSF.IO/HQSXE) that collects comments via the YouTube API, and parses the comments into a dataset while extracting additional information (e.g. user ID, timestamps, used emojis, number of likes) to prepare them for further analyses. As a potential use-case, the current version also includes a combined sentiment analysis of text and emojis.

Added Value: Emojis play an important role in YouTube comments. They can strongly influence the meaning of a comment and, thus, the associated sentiment. Nevertheless, they are rarely taken into account in automated analyses of YouTube comments. We included first solutions for the extraction and preparation of emojis for subsequent analyses in our R script. We will also provide attendants with a short tutorial on how to use the three tools we discuss in-depth (YouTube Data Tools, Webometric Analyst, and tuber package for R).

Mohseni-Methods and Tools for the Automatic Sampling and Analysis-216.pptx
12:00 - 1:00C03: Tracking Political Behaviour
Session Chair: Sebastian Stier, GESIS - Leibniz Institute for the Social Sciences, Germany
Room 149 

Predicting Political Behavior & Preferences Using Digital Trace Data

Ruben Bach1, Christoph Kern1, Ashley Amaya2, Florian Keusch1, Frauke Kreuter1, Jan Hecht3, Jonathan Heinemann4

1University of Mannheim, Germany; 2RTI International; 3Sinus Institut; 4respondi AG

Relevance and Research Question: In recent years, public opinion researchers have moved to new sources of data, especially data from the online world. For instance, researchers have analyzed the potential of replacing or supplementing survey data with information collected from Twitter, mobile devices and data from other places where people leave traces. Using data from social media instead of survey data may produce biased results, however, as individuals often promote a favorable, yet incomplete picture of their selves on social media. Other records of digital traces, such as browsing histories (i.e., domains visited) may provide a more complete picture of individuals' selves as they do not build on individuals' self-presentation in the online world. Browsing histories may reveal individual attitudes and behaviors because people tend to consume news that reinforce their existing views.

In this paper, we explore the feasibility of using individuals’ online activities to measure political attitudes and behavior. Specifically, we explore the potentials of using browsing histories and app usage to substitute traditional survey data by predicting individuals' political behavior and political attitudes from their online behavior.

Methods and Data: Members of a German commercial non-probability panel gave permission to track their browser and app usage over the four month period leading up to the 2017 German federal election. Panelists also participated in three waves of a panel survey where they were asked questions about the various ways they consume political news and information about politicians in both the offline and online world. We use these survey records to supplement the online behavioral records. Using machine learning methods, we compare predictive performance of various combinations of prediction models containing basic socio-demographic information only and/or models supplemented with records of online behavior.

Results and Added Value: Model performance does not allow the precise prediction of political behaviors and preferences, e.g., to replace survey questions (ROCs reach from .65 to .75). Yet, results may be used to target political campaigns more efficiently. Altogether, our approach allows us to learn whether political behavior and attitudes can be inferred from digital trace data, especially online browsing behavior.

The ideological dimension of vote choice response latency

Uwe Serdült1,3, Thomas Milic1,2, Salim Brüggemann1

1Center for Democracy Studies Aarau (ZDA), Switzerland; 2Department of Political Science, University of Zurich, Switzerland; 3College of Information Science and Engineering, Ritsumeikan University, Japan

Relevance & Research Question: Response latency para data stemming from online surveys are becoming more readily available and are gaining traction in the literature. We would like to contribute to their use in political science, more specifically in the realms of online survey data obtained before or in our case after a referendum vote. We aim to empirically test whether voters at the extremes of political ideologies such as the well-known economic left-right spectrum or GALTAN dimensions tend to respond faster when asked about their respective vote choice. We interpret lower response latency for vote choice as a an empirical proxy for a more solidified ideologic anchoring of voters.

Methods & Data: Response latency data was directly stored for each of the questions in the online survey automatically. Participants of this post-referendum survey were asked to recall their respective vote choice (a Yes or No question). To a certain degree we can control response latency for sensitivity due to issue salience, personality traits (OCEAN model) but also further socio-demographic and socio-economic variables. The data originate from two mixed-mode surveys (online, mail-in) held in the Canton of Aargau in September 2018 (n online survey = 453) and November 2018 (n = 463).

Results: Results from the September and November 2018 data sets show significant differences in the expected direction in a multivariate OLS regression model with vote choice response latency as the dependent variable. Not surprisingly, we see effects for control variables such as age and education. More interestingly, holding further variables constant, men, and in particular, survey respondents leaning to the political right are portraying shorter response latencies.

Added Value: a) Extending the use of para data to the case of Swiss post-referendum surveys; b) adding an overlooked element helping to further characterize voters at the extremes of the political spectrum; c) raising a new puzzle regarding gender as a variable for the response behavior in (online) surveys.

Serdült-The ideological dimension of vote choice response latency-196.pdf

How Nudges Can (De)polarize America: A Field Experiment on the Effects of Online Media Exposure

Pablo Barberá1, Andrew Guess2, Simon Munzert3, JungHwan Yang4

1Hertie School of Governance; 2London School of Economics; 3Princeteon University; 4University of Illinois at Urbana-Champaign

Relevance & Research Question:

Increasing media fragmentation and algorithmic personalization have led to persistent concerns about “echo chambers” and “filter bubbles” in online information consumption, which some fear could be a cause of increasing polarization in the mass public. However, well-known difficulties with self-selection bias are potentially more severe online. It is thus challenging to assess whether partisan media cause people to become more polarized, or if strong partisans select into congenial media.

Methods & Data:

To address these issues, we designed a pre-registered, randomized field experiment embedded in a nationally representative online panel survey (N = 1,500) in which we encouraged subsets to temporarily alter features of their information environment. Subjects in treatment groups were asked to (i) change their browser homepage, (ii) like a page on Facebook, and (iii) subscribe to newsletters, all corresponding to either Fox News or Huffington Post, two well-known outlets with a decidedly partisan slant. An additional control group received no such encouragement. We then followed up weeks later with post-treatment questions on attitudes, beliefs, and knowledge. Using linked data on respondents’ web visits (i.e. URL-level browsing data), which respondents agreed to provide via software that they installed on their devices, we are able to precisely measure treatment effects among compliers and gauge the extent to which their information-seeking habits were altered over time.


The field experiment was launched in October 2018. We have published our pre-analysis plan online and will be able to start analyzing the data within the next weeks. Given that our hypotheses and strategy to analyze the data are fixed and documented, we will be able to present detailed results at GOR in March.

Added Value:

We hope that our research will shed light on the power of relatively small “nudges” in online choice architecture to affect people’s media consumption behavior as well as longstanding attitudes and beliefs. We offer an experimental design that allows disentangling selection effects from actual exposure effects. Our panel design setup provides opportunities for meaningful pre-treatment post-treatment comparisons. Finally, the tracking data on respondents' browsing behavior allows measuring information-seeking behavior at previously unknown precision.

12:00 - 1:00D03: GOR Best Practice Award 2019 Competition II
Session Chair: Otto Hellwig, respondi AG & DGOF, Germany
Session Chair: Alexandra Wachenfeld-Schell, GIM Gesellschaft für Innovative Marktforschung mbH & DGOF, Germany
Room 248 

Impact of subscription and discount cards on mobility decision-making: the example of BahnCard in the NRW tariff

Andreas Krämer1, Till Ponath2, Hans Dethlefsen3

1University of Applied Sciences Europe, Germany; 2Kompetenzcenter Marketing NRW; 3DB Fernverkehr AG

Relevance & Research Question:

In 2005, the NRW tariff for "cross-border" journeys was introduced. Since then, there has been a door-to-door tariff for all journeys with buses and trains in NRW transport. Owners of a BahnCard (25/50) receive a discount on the price of the single relational ticket. While the BahnCard was developed for the market for long-distance train journeys (> 5 Mio. cards), its use and economics in regional transport are highly controversial.

Methods & Data:

Key component of the project was a survey of ticket users. Due to difficult framework conditions (low incidence in trains of 0.04 %, little specific knowledge etc.), the project defined that firstly the study was conducted as an online survey (April 2017, n = 3.554) and secondly, that the survey should be based on CRM data from the Deutsche Bahn loyalty program (BahnBonus). These played a crucial role in different phases of the market research process: (a) recruitment: use of contact data, (b) interview, focus on actual journeys with BahnCard discounts (c) analysis: validation of the response behavior and (d) extrapolation: revenue effects (with/without BahnCard) concerning the population of journeys (1.1m in 2016).


The empirical study indicates that the BahnCard discount in the NRW tariff is not a key driver for the BahnCard purchase decision. However, owning a BahnCard and receiving a discount lead to a demand shift in favor of the regional train. BahnCard acceptance in the NRW tariff, overall results in a positive revenue effect of more than EUR 4 million p.a. for regional transport - customers benefit from low transport costs, making the rail system more competitive.

Added Value:

CRM-data-supported online surveys can be prerequisite for detailed customer insights. The presented project example leads to clear findings that hardly be possible based on conventional CATI or F2F interviews. Besides the core project findings, additional results provide a better understanding of the effects of subscription models emerge that goes beyond common explanations (sunk cost, subscription biases). Our empirical study support the idea that “Sunk Costs” can be quite relevant for the decision-making behavior of consumers, however this is not necessarily irrational.

Automation of the Real Voice of the Customer. Use of massive audio and video interaction in online interviews

Holger Lütters1, Malte Friedrich-Freksa2, Dmitrij Feller3, Marc Egger4, Mark Wolff2

1HTW Berlin, Germany; 2GapFish GmbH, Germany; 3pangea labs GmbH, Germany; 4Insius UG, Germany

Voice is a new interaction mode in the digital environment supported by players like Amazon, Google, Microsoft etc. The very recent development of devices also allows video interaction.

A team of researchers was inspired by the interaction and automation opportunities of the new technologies and developed a study testing new interaction modes in comparison with the established mode of typing open answers with a keyboard.

A device agnostic BYOD approach was developed by pangea labs to take advantage of the existing devices in the hands of consumers. The technology was integrated in a web-browser survey which is functional on the main categories Desktop, Laptop, Tablet and Smartphone. The charm of this initiative lies in the detail of the direct access to the market research infrastructure which can be used without additional cost.

A massive study with 9813 panelists of the German panel EntscheiderClub was conducted. After checking for technical feasibility and willingness to participate 1251 interviews (13 Minutes 20 seconds median) were conducted in 3 random routes:

Classical keyboard (Control group n=437)

Voice interaction (Experimental group 1 n=407)

Video interaction (Experimental group 2 n=407)

The study was using 9 open ended interactions to be compared between the control group and the two experimental groups with the topic “digital assistants in the household”.

In total 7326 audio and video responses were collected. The innovative part of this research was the development of fully automated transcription tools using Natural Language Processing to create transcripts of the recorded audio and video footage.

The data collected was transferred to the reporting partner insius analyzing the formerly spoken words. The results show a much more intense feedback of the respondents using the experimental conditions. People’s willingness to express themselves grows by 250% on average using voice instead of typing. With video the amount of answers grows by 150% compared to the well established mode of typing in answers with a keyboard.

The virtual team hopes to be a candidate for the best practice session at GOR19 as we see this joint project as a role model of expert's cooperation all along the future value chain of market research.

Revolution of the VW customer journey

Nina Bethmann1, Artur Kryzan2

1InSites Consulting, Germany; 2Volkswagen Poland

Relevance and Research question:

Volkswagen, the German car manufacturer founded in 1937, has always been leading in the car industry. Yet in recent years, the (perceived) quality gap with other car brands has decreased due to high fragmentation and innovation in the market. Volkswagen thus wanted to see on what other levels it could (re)claim its position in the market. The brand embarked a mission to find unique customer experience solutions to set the brand apart and to boost conversion by digitalizing the pre-and after customer journey.

Method & Data:

To extract the right insights that should lead to an unraveled, intuitive & seamless customer experience by rolling out new CX solutions the project was set up in four stages: harvest, define, ideate and prototype.

Based on dealer interviews and industry reports the existing knowns were harvested and success criteria were defined. Through an online community (called the VW Square) of 105 (potential) VW customers the whole journey was mapped out identifying customer needs across all relevant moments. Engaging tasks provided deep consumer understanding throughout all journey moments, resulting in 10 universal need states and 26 jobs to be done. In a next phase, ideas were crafted by sourcing different crowds: ideation with consumers, team brainstorms and crowd sourcing with creatives in an Eyeka community bringing more out-of-the-box propositions. After a validation phase, we embarked an agile and iterative prototyping phase where concepts were tested amongst consumers and fed back to the business.


This iterative, agile development of customer service solutions resulted in different prototypes which are currently being tested (with success!) at the Volkswagen Home flagship store in Warshaw and planned for roll-out as of 2019.

Added Value:

Consumer centered customer journey mapping enabled VW to detect insights that allowed for cutting-edge CX solutions. It also offered a CX strategy canvas for CX innovation in the coming years, empowering VW to take a steep learning curve and becoming self-sustainable in developing and rolling-out new CX solutions. The staged approach, involving a broad set of stakeholders, formed the start of a transformation process towards a truly consumer centered organization.

1:00 - 2:00DGOF-Mitglieder-Workshop: Repräsentativität in der Online-Forschung - wie kann das gelingen?
Session Chair: Alexandra Wachenfeld-Schell, GIM Gesellschaft für Innovative Marktforschung mbH & DGOF, Germany
Session Chair: Holger Geißler, DCORE GmbH & Datalion GmbH, Germany

Dieser Workshop ist nur für DGOF-Mitglieder!
Für das leibliche Wohl sorgt respondi.
Room 147 
1:00 - 2:15Lunch Break
2:15 - 3:30E: Spotlight Global Research Quality Standard ISO 20252

This talk ends at 3pm
Room 69 

Why do so many companies and institutions fail despite a working business model?

Olaf Hofmann

SKOPOS, Germany

How a strong focus on quality and processes contributes to the success of companies like Amazon - and why an ISO 20252 certification should therefore be at the core of your business focus, too.

Companies like Amazon and many others in the digital space have successfully proven that a strong focus on processes, customer orientation, and quality will lead almost automatically to strong and steady growth and a healthy bottom line.

Many research companies, old ones and new ones, are lacking this focus. This has an increasingly negative impact on their businesses – and it helps new players and startups tremendously. Digitalization further increases negative impact of a lacking focus on quality.

The global research quality standard ISO 20252 has been developed to assist research providers and end clients to find and / or to maintain their focus on quality and customer orientation.

Specific examples on how the implementation of ISO 20252 have changed research organizations will be provided. And we will see how becoming customer and therefore more quality focused ultimately leads to significant improvements in their market success.

Olaf Hofmann has studied Psychology at the University of Bonn and has founded SKOPOS in 1995. Olaf is participating in the German ISO mirror committee since 2002 and is a member of the ISO-Delegation which represents Austria, Swiss and Germany since 2005.

Hofmann-Why do so many companies and institutions fail despite a working business model-260.pdf
2:15 - 3:30F 1: Poster Session (Part I)

Finding the trolls lurking beneath the news. A two-step approach to identify perceived propaganda through machine learning.

Vlad Achimescu

University of Mannheim, Germany

Relevance & Research Question: Recently, numerous attempts by foreign actors to manipulate public opinion were uncovered, where false accounts are employed to spread propaganda online. Eastern Europe is highly exposed and vulnerable to this type of political astroturfing. Users of online newspaper forums have been vocal in calling out some posters as ‘Russian trolls’, in an act of informal flagging. I investigate the potential of using these informal flags to predict perceived propaganda using machine learning models in a two-step approach.

Methods & Data: Over 200.000 comments posted to articles published in 2017 on a large Romanian online newspaper were scraped. Using specific keywords and manual classification, informal flags are identified. Supervised machine learning (regularized logistic regression and random forests) is used in two steps. The first step predicts whether a comment is an informal flag or not, based on the word content and metadata (Model 1). In the second step, flagged messages are labeled as potential propaganda and another model predicts whether a message would be flagged or not (Model 2) using the same features as Model 1.

Results: Through manual classification, 350 informal flags are identified. The best model in the first task has a precision of 0.69 on the test set. Applying this model to unlabeled data, 430 additional informal flags are discovered. Using both initial and additional flags in Model 2 improves prediction accuracy from 0.76 to 0.85, compared to using initial flags only. Random forests show improved performance over regularized logistic regression. Word content is key for identifying flags, while metadata is essential for identifying messages posted by trolls. Informal flaggers write shorter messages that get positive ratings, while trolls tend to obtain more negative ratings.

Added Value: This research contributes to the identification of online propaganda using computational text analysis. It shows the potential of externalizing the process of labeling to members of online communities, but it also highlights the risks of misclassification. The improved accuracy of the two step approach shows that it is necessary to periodically update the labeling process rather than to rely on a fixed model.

Achimescu-Finding the trolls lurking beneath the news A two-step approach-280.pdf

Do We Blame it for Its Gender? How Specific Gender Cues Affect the Evaluation of Virtual Online Assistants

Carolin Straßmann, Annika Arndt, Anna Dahm, Dennis Nissen, Björn Zwickler, Bijko Regy, Melissa Güven, Simon Schulz, Sabrina Eimler

Hochschule Ruhr West, Germany


Virtual online assistants give us recommendations on websites or help us with our daily lives. These agents have mostly a humanoid design and are associated with a gender. Based on the media equation theory (Revees & Nass, 1996) assistants trigger the same social responses as humans. Consequently, gender stereotypes are applied, which were found to affect the perception of the agent (c.f. Nowak & Fox, 2018). The gender of the assistant can be conveyed by different cues, which might make these stereotypes more or less salient. The present study aims to investigate the effect of different cues representing assistant’s gender on its evaluation.


An online experiment with a 2x3x2 between-subjects-design, where gender (male vs. female), gender cues (either represented by name, embodied character or voice) and interaction quality (flawless interactions or incorrect interaction) was conducted. A total of 138 people (52 female; Mage = 23.93, SDage = 8.19) completed the questionnaire. Participants evaluated the assistant afterwards with regard to warmth and competence.


Results indicate that female assistant were perceived as warmer than male (F(1, 135) = 4.58, p = .034, η2 = 0.03) and that when the interaction was flawless, the agent got evaluated as more competent after a flawless interaction than after an incorrect interaction (F(1, 135) = 4.07, p = .046, η2 = 0.03). Moreover, the representation of the gender differed with regard to warmth (F(2, 135) = 4.76, p = .010, η2 = 0.07), where the voice was perceived as significantly less warm than the name or embodied character. Additionally, a 3-way interaction between all independent variables occurred with regard to warmth (F(2, 135) = 3.20, p = .044, η2 = 0.05): For female agents represented through an embodied character an interaction with a failure leads to a higher warmth evaluation than a flawless interaction.


The studies’ findings emphasize that gender stereotypes and their consequences are deeply rooted in the human’s nature. Moreover, specific representation of the assistant’s gender seem to boost the application of gender stereotypes.

Straßmann-Do We Blame it for Its Gender How Specific Gender Cues Affect the Evaluation-285.pdf

Teaching Practical Tasks with Virtual Reality and Augmented Reality: An Experimental Study Comparing Learning Outcomes

Alexander Arntz, Sabrina Eimler, Uwe Handmann

Hochschule Ruhr West, Germany


Currently the effectiveness of Virtual Reality (VR) and Augmented Reality (AR) systems as teaching methods for practical skills is largely unexplored. Studies exploring the question whether these systems can provide the same or better learning outcomes than a text instructed practical task are still missing. This abstract describes result from an experimental study exploring computer assembling tasks combined with pre-/post-online questionnaire.


Three conditions (VR, AR and a real setup) were used to teach participant how to assemble a standard desktop computer. Each condition was divided into two parts: (1) participants were confronted with their specific scenario, (2) participants had to go through a real practice after one week. The experimental setup was accompanied by pre- and post-condition-online-questionnaires (using SoSciSurvey). Besides performance data (i.e. learning outcome), wellbeing, prior knowledge of the task and the system used as well as system usability measures were assesses. The survey helped to determine the learning outcome by containing a quiz that queried the designation, function and the correct assembling of the components. Time required to complete the task and error quote were collected using a checklist.


Results concerning the learning outcome showed that participants in the VR-condition outperformed those who learned from the real setup ((M=10.0, SD=0.0) [virtual reality] vs. (M=8.95, SD=1.27) [control]). Furthermore, results from the assembling duration assessment demonstrated that the VR-group participants completed their tasks 6.62% faster than the control group. Regarding the identification of hardware parts, both groups had a significant improvement during the post-condition compared to the first test run, indicating a learning progress. However, due to the VR group achieving a better outcome in average answers and a more significant difference between the trials, the results indicate a better performance by participants assigned to the VR-condition.


The results show that VR and AR systems could exceed text-based approach in terms of learning outcome performance. The effectiveness of the systems implicates a major benefit for the educational landscape, as learning content that is not realizable in terms of cost, distance or logistics could be designed as an immersive and engaging experience.

Arntz-Teaching Practical Tasks with Virtual Reality and Augmented Reality-284.pptx

Web Survey on e-grocery consumers’ attitudes- An efficient design experiment that mixes stated preference and rating conjoint tasks.

Orlando Marco Belcore1, Luigi Dell'Olio2, Massimo Di Gangi1

1Università degli Studi di Messina, Italy; 2Universidad de Cantabria, Santander, Spain

Relevance & Research Question: Digital infrastructures have changed everyday life, thereby helping us to solve different tasks. The e-grocery represents a new barrier for e-commerce, so a web-based survey has been developed with the aim to: reach very diverse samples, intercept consumers’ feelings, understand their perceived value and provide information on real behaviours. This proposal would represent an effective instrument to evaluate the future demand of e-grocery services and the impact generated by these on urban areas.

Methods & Data: The proposed web survey consists of three fundamental sections: a revealed preferences (RP) one, an efficient experimental design as Stated Preference (SP) and a rating based conjoint task. To help people who are not familiar with e-grocery and choice experiments, multimedia contents have been developed inside the web site and the survey. Moreover to overcome limitations of SP experiments when complex situation has to be studied, the scenarios have been divided in three steps and variables introduced combining images and descriptions creating an artificial purchase timeline that help the interviewed to handle a wider range of variables by solving simple tasks.

Starting from January 2019 respondents has been recruited spreading the survey by means QRcode touchpoints and social media.

Results: Submitting the survey to experienced consumers and newcomers across countries helps a more realistic evaluation increasing reliability of data. The recursive choice task inside the SP let to evaluate single variable relative weight and its cut off points. Data from Likert evaluation are used to strength the reliability of SP experiment pointing out the existence of patterns and situations that bring decision makers to select a specific purchase strategy increasing so clustering flexibility for analysts.

Added Value: This experiment, representing for an interviewed a complete cognitive process, underlines the potential offered by supplementing data from random utility theory and conjoint analysis to evaluate consumers’ attitudes, expectations and choices. The introduction of a multi-channel purchase option overcomes the limitation to agree or not with online strategy and the “timeline” solution increases reliability meets the satisfying of simplicity and accuracy. This will allow us to strength classical latent class models.

Belcore-Web Survey on e-grocery consumers’ attitudes- An efficient design experiment that mixes stated p.pdf

When Gender-Bias Meets Fake-News - Results of Two Experimental Online-Studies

Sarah Bludau1, Gabriel Brandenberg2, Lukas Erle2, Sabrina Eimler2

1University of Duisburg-Essen, Germany; 2University of Applied Science Ruhr West, Germany

RELEVANCE & RESEARCH QUESTION: Online media are perceived to be credible sources for information and up-to-date news. However, this also implies new possibilities to publish false information accessible to a large population. Information are evaluated unconsciously, resulting in biased interpretations and attributions. Goldberg (1968) demonstrated that female authors are perceived less credible than identical articles written by male authors. As a result gender-stereotypes could facilitate higher credibility of false information in online-settings. It is assumed that these biases also apply in online media with a variety of consequences for individuals and society.

METHODS & DATA: In two online studies (N = 226; N = 95) four stimulus articles were presented in a 2 (male vs. female author of text) x2 (reported misusage of a technical innovation by men vs. women) between-subjects-design. Participants were assigned randomly to one experimental condition and asked for their perception of the text (e.g. quality, style, credibility) and author (e.g. warmth, competence). The second study considered perceived authenticity in addition.

RESULTS: Results yield a main effect of authors gender on perceived credibility F(1, 224) = 5.04, p < 0.05, η2 = 0.05 showing higher scores for male authors. Participants considered articles presenting male misusage of a technical innovation to be less credible (F(1, 224) = 4.54, p < 0.05, η2 = 0.02). Also, there is an interaction effect showing that articles describing female misusage of technical innovation written by a male author are evaluated most credible (F1, 222) = 4.01, p < 0.05; η2 = 0.02), also confirming the assumption of a gender-bias in online media. Female authors’ warmth was perceived higher than males’ (F(1,224) = 11.08, p = .001, η² = .05), whereas no difference was found regarding perceived competence of the author. The second study showed that perceived authenticity has an impact on authors rating.

ADDED VALUE: Results indicate that, despite a change in the prevalence of female authors (e.g. bloggers, influencers) in the internet, a certain reproduction and stability of gender stereotypes still exists. Also, gender biases are at least partially intertwined with news credibility. Further results and limitations will be discussed.

Bludau-When Gender-Bias Meets Fake-News-283.pdf

Making Online Research Findable, Accessible, Interoperable and Reusable (FAIR)

Ines Drefs

GO FAIR International Support & Coordination Office, Germany

Across all disciplines, research is faced with digitalization and ensuing expectations towards sharing digital(ized) data, especially when publicly funded. In the field of online research, data are digital and thus machine-readable by nature. Hence, not only expectations are high in terms of full exploitation of this research data. There is also increased potential for data sharing and re-use within the field of online research. For researchers to manage their data, the so-called FAIR principles have been widely promoted as guidelines implying that research data should be made *f*indable, *a*ccessible, *i*nteroperable and *r*eusable. How can data FAIRification be best realized in online research? How can online researchers benefit from synergies when developing solutions for FAIR data management?

In the transition to FAIR, early movers from various disciplines and regions have started to organize themselves as so-called Implementation Networks (INs) of the GO FAIR initiative. GO FAIR INs consist of individuals, institutions and organisations committed to making services and data FAIR. At a practical level, this happens on the basis of three interactive processes which constitute the pillars of GO FAIR: GO CHANGE refers to IN activities that foster a socio-cultural change toward data sharing in the broader scientific system. The GO TRAIN process is fostered by INs who develop training curricula focused on FAIR Data Stewardship as well as certification schemes for pertinent competencies. GO BUILD refers to INs’ efforts of developing technical standards and infrastructure components needed to create an Internet of FAIR Data and Services.

Since its kick-off in 2018, GO FAIR has seen the emergence of more than 30 Implementation Networks, now in various stages of development. Collectively, the INs span a broad range of actors including research communities, service providers, librarians and funders.

GO FAIR participants benefit from workshops and meetings for knowledge exchange and knowledge transfer organized by the initiative’s Support and Coordination Office. By synchronizing their “FAIRification” efforts, the INs create synergies and avoid fragmentation as well as silo formation. INs can be joined any time or new INs can be launched. As such the GO FAIR initiative is entirely open, inclusive and stakeholder-driven.

Drefs-Making Online Research Findable, Accessible, Interoperable and Reusable-269.pdf

Fightclub - Market research vs. UX research

Lisa Dust1, Christian Graf2

1Facts and Stories GmbH, Germany; 2UXessible GbR, Germany

Relevance & Research Question: The relationship between UX research (user research as part of the user experience design) on the one hand and market research on the other hand is being discussed again and again lately. While one part of the community tends to emphasize differences between the two fields, the other points out clear overlaps. One only seems to agree that market and UX research is important. So, in the view of the protagonists, what is the relationship between both and what role does each play in the typical lifecycle of product? Where are the differences and commonalities?

Methods & Data: An online survey with closed and open questions was started. N=37 professionals completed it.

Results: We found that relevance of market research is seen as especially strong in the research and market entry stage (not surprisingly). UX research is regarded as especially strong in the conceptual and implementation stage. Interestingly when looked over all stages both are complementing each other.

Added Value: Lisa as a representative of market research and Christian as the one with UX research experience have embarked on a journey of discovery. They present controversial points for you and report on their (provisional) results. They want to enable a discussion, so that everyone has a better idea about each other's strengths and how to use them adequately.

Dust-Fightclub - Market research vs UX research-286.pdf

Survey Attitude Scale (SAS): Are Measurements Comparable Among Different Samples of Students from German Higher Education Institutions?

Isabelle Fiedler, Ulrike Schwabe, Swetlana Sudheimer, Nadin Kastirke, Gritt Fehring

Deutsches Zentrum für Hochschul- und Wissenschaftsforschung (DZHW), Germany

Besides others, general attitudes towards surveys are part of respondent’s motivation for survey participation. There is empirical evidence that these attitudes do predict participant’s willingness to perform supportively during (online) surveys (de Leeuw et al. 2017; Jungermann/Stocké 2017; Stocké 2006). Hence, the Survey Attitude Scale (SAS) as proposed by de Leeuw et al. (2010) differentiates between three dimensions: (i) survey enjoyment, (ii) survey value, and (iii) survey burden. Referring to de Leeuw et al. 2017, we investigate into the question whether the SAS measurements can be compared across different online survey samples of students from German Higher Education Institutions (HEI).

Therefore, we implemented the nine item short form of the SAS, adopted from the GESIS Online Panel (Struminskaya et al. 2015) at the beginning of three different online surveys for German students and PhD students being conducted recently: First, the HISBUS Online Access Panel – a periodic cross-sectional study of higher education students on current study specific issues (winter 2017/2018: n=4,895), second the seventh online survey of the National Educational Panel Study (NEPS) - Starting Cohort “First-Year Students” (winter 2018: n=4,939), and third, a quantitative pretest among PhD students within the National Academics Panel Study (Nacaps; spring 2018: n=2,424). To validate the original scale in each dataset we use confirmatory factor analysis (CFA).

Comparing the CFA results, our empirical findings indicate that the latent structure of the SAS is reproducible in all three samples. Factor loadings as well as reliability scores support the theoretical structure adequately. Thereby, our findings support the validity of the proposed nine item short form of the SAS, for new and repeated respondents as well.

By showing that the standardized short SAS instrument works for different samples, we contribute to existing literature. Since de Leeuw et al. 2017 analyses are based on four general population surveys, we complete the picture specifically for young highly educated respondents. For further research, we aim to pool our data to investigate into more sophisticated methods ensuring measurement equivalence (Chen 2007).

Fiedler-Survey Attitude Scale-291.pdf

Embedding the first question in the e-mail invitation: the effect on web survey response

Marco Fornea1, Chiara Respi2, Beatrice Bartoli1, Manuela Ravagnan1

1Demetra srl, Italy; 2University of Milano-Bicocca, Italy

Relevance&Research Question: Low response rates in web surveys are a challenging issue. Researchers explore several response inducements, e.g. when inviting participants through e-mail. However, we are aware of only two studies that focus on embedded questions in the e-mail invitation. The main aim of this poster is to assess the impact of tailoring the e-mail invitation text on response. In particular, we evaluate the impact of an e-mail invitation that includes the first question of the web questionnaire vs a standard e-mail invitation on survey participation, questionnaire completion and completion time, break offs, and respondents’ composition.

Methods&Data: We use experimental data from a web survey conducted on delegates of the trade union “Italian General Confederation of Labour”. Sample members (N=5,494) were stratified by geographic area and type of trade-union category, and then they were randomly (within the strata) assigned to two groups: the “link” and the “first question” e-mail invitation group. The text of the e-mail sent to the two groups was different only in the final statement. At the end of the e-mail text (the same for both groups), in the “first question” group, the first question of the questionnaire was reported, while, in the “link” group, the survey link was included. To analyse our data we adopt both bivariate and multivariate analysis.

Results: Preliminary findings show that the “first question” group is more likely to complete the questionnaire than the “link” group. The higher break-off rate for the “first question” group suggests that the embedded invitation is also effective in stimulating “reluctant” respondents to start the questionnaire. Moreover, there are no significant differences between the two groups on completion time. Lastly, respondents from the geographic areas where Internet access is less spread are more likely to respond when invited through an embedded e-mail.

AddedValue: We believe that our work may contribute to expand the knowledge on the effectiveness of embedding a question in the e-mail invitation on response. Indeed, to the best of our knowledge, this is the first study that looks at the impact of the embedded invitation on completion time and respondents’ composition.

Fornea-Embedding the first question in the e-mail invitation-264.pdf
2:15 - 3:30F 2: Poster Session (Part II)

Selection Bias and Representativeness of Survey Samples: the Effectiveness of Mixing Modes and Sampling Frames

Beatrice Bartoli1, Chiara Respi2, Marco Fornea1, Manuela Ravagnan1

1Demetra, Italy; 2University of Milano-Bicocca, Italy

Relevance&ResearchQuestion: Nowadays mixed-mode approaches are used to deal with the non-coverage issue in sample surveys. There are many examples of surveys that mix web, telephone and F2F modes, often using the same sampling frame. Drawing on our previous work, we apply a mixed-mode survey design to different sampling frames (landline phone list and online panelists). We found that telephone coverage bias may be reduced adopting approaches that use different sampling frames. This paper aims to study the representativeness of samples from a mixed-mode survey design (web-landline phone) and from a telephone survey (calling mobile phone and landline phone numbers), comparing their estimates to the Italian population’s characteristics and to the observed values from registered voters’ records.

Methods&Data: We use data from 5 telephone and web surveys conducted in Italy from March 2018 to December 2018 on landline or mobile phone owners and on members of an Italian online panel. We designed a mixed-mode survey (a Computer Assisted Web Interview - CAWI survey followed by a Computer Assisted Telephone Interview - CATI survey, using two different sampling frames) and a survey with two different sampling frames (a Computer Assisted Mobile phone Interview - CAMI survey followed by a CATI survey). To study the representativeness of the samples, we compare the estimated vote behaviour from the two survey designs to the observed values of vote behaviours in the recent National elections (in March 2018). We also compare the employment status and education of all respondents to a “gold standard”.

Results: Results from our previous work are confirmed. Indeed, we find that mixing both modes and sampling frames is more effective in reducing selection bias than mixing sampling frames only. In our analysis, the CAWI-CATI samples perform better than the CAMI-CATI ones in estimating vote behaviour and employment status of the Italian population. Both CAWI-CATI and CAMI-CATI respondents are more educated than general population.

AddedValue: Our poster contributes to expand the knowledge on mixing modes and sampling frames to reduce bias. The main value of this work is the large number of public opinion surveys we added to those conducted two years ago, providing robust findings.

Bartoli-Selection Bias and Representativeness of Survey Samples-263.pdf

“Ok google” - The role of digital Voice Assistants in the lifeworlds of users - An empirical study on relationship types between Voice Assistants and users

Anna Kaiser2, Ivonne Preusser1, Janine Bunzeck1

1TH Koeln, Germany; 2Skopos Connect, Germany

Relevance & Research Question: Alexa, Siri and Google Assistant have found their way into the personal privacy. According to a forecast, global user numbers are expected to rise from 1,376 million (2019) to 1,831 by 2020 (Tractica 2018). The ability to imitate and understand language creates a social interaction between human and machines.

This study examines the question of whether there is a human-like relationship between digital Voice Assistants and users and which types can be derived from this. Differences in the relationships will be explored and central features within these will be identified.

The study is based on the concept of Media Equation (Reeves, Nass 1996), which examines influencing factors of humanization and behavior patterns towards digital media. Theories of attachment (Bowlby 1969) and social network (Granovetter 1973) can be transferred to social concepts of technology.

Methods & Data: We implemented an online community platform with a mixed-method design. The Study is divided in two online survey periods, of each 14 days. The explorative design contains 14 diary entries on a private blog and 11 tasks with open questions on a survey platform. A system board (Kaspersky Lap 2016) and the BFI-10 (Rammstedt et al. 2013) are part of it. The field phase was flexible and contained a daily diary as a central theme. Additionally 2 video interviews were conducted. In sum data of 54 participants (54 % female) was generated.

Results: The results underline the theory of media equation with regard to Voice Assistants. Three different types of relationships between the users and the Voice Assistants could be identified. These types differ essentially in degree of humanization of the device and its integration into the user's lifeworld’s. From these findings it can be concluded that Voice Assistants for some users represent more than just a technical device.

Added Value: The knowledge gained for science consists of creating parallels and links to areas that have already been researched more intensively, such as computer research of human-machines interaction and to expand social concepts of technology. It remains interesting to observe the development of relationships with Voice Assistants as technology progresses.

Kaiser-“Ok google” - The role of digital Voice Assistants-292.pdf

What Predicts the Validity of Self-Reported Paradata? Results from the German HISBUS Online Access Panel

Nadin Kastirke, Swetlana Sudheimer, Gritt Fehring, Ulrike Schwabe

German Centre for Higher Education Research and Science Studies (DZHW), Germany

Paradata, such as user agent strings (UAS), provide us with important client-side information about the technical conditions of web surveys. If due to different reasons for example data protection issues UAS are not available, one may directly ask survey participants for the required data. Until now it is unclear whether the validity of these self-reported paradata is determined by participants’ general attitudes towards surveys, their willingness to participate and – in connection with technically demanding questions – distraction while answering.

To shed light into the question what predicts the consistency between UAS and self-reported paradata, we use the HISBUS Online Access Panel. The sample comprised 3,137 members with UAS known to us that were asked for used device (DEV), operating system (OS) and web browser (WB). Additionally, data on general attitudes towards surveys (SAS; de Leeuw et al. 2010), survey participation evaluation (SPE; Struminskaya et al. 2015) and multitasking (MT; Zwarun/Hall 2014) were collected. The Big Five personality traits (BF; Rammstedt et al. 2013) serve as covariates in our applied logistic and ordinal regression analyses for DEV as well as OS/WB, respectively. Predictors with p<.05 were included in the final models.

First of all, UAS and self-reported paradata were highly consistent (kappa: DEV=.95, OS=.95, WB=.87). Agreement regarding DEV depends on SAS subscale value (OR=1.31) and SPE (OR=0.80). The OS/WB agreement was predicted by electronic and non-electronic MT (OR=1.32; OR=0.72).

We conclude that directly asking web survey participants is a promising way to get valid information about their technical equipment if UAS data are not available. The chance to get valid DEV data is higher if surveys are generally considered valuable, but lower if the evaluation of willingness to participate is considered to be solid. The chance for valid OS/WB data is higher with electronic MT present that may indicate technical skills. Non-electronic MT seems to be rather distracting and predicts lower chances for valid OS/WB data.

Kastirke-What Predicts the Validity of Self-Reported Paradata Results-268.pdf

Working towards understanding and enhancing Enterprise Social Network use

Lena S. Kegel1, Martin Salaschek2, Meinald T. Thielsch1

1Westfälische Wilhelms-Universität Münster, Germany; 2Federal Centre for Health Education (BZgA), Germany

Relevance & Research Question:

Enterprise Social Networks (ESN) are established by many organisations in order to promote consumption of and contribution to knowledge among their employees. Despite high costs and effort to implement an ESN, many fail as a consequence of low user participation. Users‘ missing ability to translate usage intentions into specific use cases is regarded as a major reason for this outcome (Chin, Evans, & Choo, 2015). In an experimental study, we examined whether strengthening users‘ ability to use ESN by boosting self-efficacy and using Implementation Intentions are possibilities to enhance employee participation.

Methods & Data:

The field experiment was conducted with a sample of users of inforo, an online community for health professionals run by the German Federal Centre for Health Education (BZgA). 63 participants (mean age 48.10 years; 44 women) were randomly assigned to the conditions self-efficacy and Implementation Intentions (II) in a 2x2 study design. Self-efficacy was promoted by adding supporting formulations in the invitation mail to the study and by adding an explanatory video about the ESN. In the II condition, participants were instructed to formulate Implementation Intentions in comparison to mere intentions in the non-II condition. For each participant, ESN-use two weeks after participation was used as outcome measure (objective behavioral data from a dedicated logging tool).


Manipulation of self-efficacy in this study was successful (r = .34). However, neither self-efficacy nor Implementation Intentions nor their interaction could significantly explain ESN-use. These findings indicate that either users‘ ability has no influence on ESN-use or it cannot compensate for other influential factors. The latter explanation is supported by the overall low user participation in the ESN, which indicates hindrances that are independent of individual-centred factors.

Added Value:

The findings show that enhancement of ESN-use needs to be regarded in a broader context of human-computer interaction. Besides individual-centered variables, further research should focus on organizational, technical and social factors to extend our understanding of ESN-use. Using objective online tracking data - as it was used in this study - should be continued in this field of research to create relevant information for practitioners.

Kegel-Working towards understanding and enhancing Enterprise Social Network use-276.pdf

Developing Podcasts that Inspire Listeners and Facilitate Learning

Lars König

University of Münster, Germany

Relevance & Research Question: Teacher enthusiasm can be defined as the occurrence of distinct behavioral expressions, such as nonverbal (e.g., gestures) and verbal (e.g., tone of voice) behaviors. It has been shown that teacher enthusiasm is linked to various positive outcomes: It is linked to students’ enjoyment, interest, achievement, motivation and vitality. However, most teacher enthusiasm research is based on correlational data and therefore no causal inferences can be drawn.

Methods & Data: To overcome this limitation, a between-subject experimental design was used to analyze the effects of teacher enthusiasm on instructional quality. Two versions of an evolutionary psychology podcast were developed: A neutral and an enthusiastic version. While the wording was kept identical between both versions, the speaker was instructed to read the podcast script either in a neutral or in an enthusiastic manner. It was hypothesized that listening to the enthusiastic version would result in more positive instructional quality ratings. University students with diverse majors listened to the podcast. To test the hypothesis, independent sample t-tests were conducted.

Results: Overall, the results show that listening to the enthusiastic version resulted in more positive instructional quality ratings: Participants who listened to the enthusiastic version of the podcast rated it as more interesting and exiting; they enjoyed the listening process more; had a higher motivation to learn more about the topic; evaluated the podcast host as more trustworthy; and gave the podcast a higher overall rating.

Added Value: The results demonstrate that teacher enthusiasm can be a powerful instructional tool when developing educational podcasts.

König-Developing Podcasts that Inspire Listeners and Facilitate Learning-265.pdf

„Eggs“-plaining Differences in Market Share and Optimal Pricing – a Comparison of Online Methods

Carina Krämer, Daniel Althaus, Kolja Turkiewicz, Luise Neumann


Data science is incurring ever-larger parts of market research. A large part of that success is due to the ability to predict future sales from past user behavior. Market research could counter with its ability to predict future sales from individual preferences, making it possible to analyze potential success even before putting a product to market and preventing expensive flops. For this purpose, market research employs experimental methods, namely price sensitivity measurement (PSM) and choice-based conjoint analysis (CBC), but often asks directly for purchasing preferences. This poster compares willingness to pay and market share for eggs from organic, free-range and barn poultry farming in Germany, generated by PSM, CBC and direct questions. It is based on a quantitative online study with 1.011 interviews with participants between 18-69 years of age, sampled representatively for gender, age and region from members of the German online access panel of Splendid Research.

When queried directly, consumers tend to overestimate their propensity to buy organic or free-range produced eggs. This results in greater expected market shares for the ethically correct products. Calculating market shares based individual utilities derived from choice-based conjoint analysis produces almost exactly the market shares determined by the validation question and can be considered extremely valid. The optimal prices for organic eggs determined by PSM and both CBC variants come close to the actual market prices. The optimal price for free-range eggs in PSM is much lower than the actual price at EDEKA and the price arrived at by both conjoint models. All three models indicate much higher optimal prices for barn eggs than the ones in place. The lower prices for barn eggs exist probably due to bait pricing, as inputting the current prices leads to very similar market shares.

All in all, market research’s, and especially online market research’s experimental methods can be trusted to provide good estimations of unknown quantities if used the right way. Unreflected and simplified questions can cause large bias in the estimation of those same quantities. The poster therefore advocates the use of experimental designs and creative validation techniques in online surveys.

Krämer-„Eggs“-plaining Differences in Market Share and Optimal Pricing – a Comparison-288.pdf

The impact of a mobile option in a migration survey on sample composition and data quality. Results from a multilingual feasibility study.

Ilter Öztürk, Johannes Lemcke, Marie-Luise Zeisler, Patrick Schmich, Claudia Santos-Hövener

Robert Koch Institut, Germany

Relevance & Research Question:

A common new method of generating survey data is via online mobile devices. Especially in so called hard-to-reach populations (e.g. people with migration background) the implementation of a smartphone survey option can increase the response. However; there are methodological challenges: The usage of different devices to complete a survey could lead to device effects, compromising the data quality. Utilizing a survey among migrant populations in Germany, our objective is to identify determinants influencing the usage of mobile devices and to investigate differences in the resulting data quality between desktop and mobile devices.

Methods & Data:

We used data from a multilingual feasibility study that was conducted in two German federal states, utilizing data from residents’ registry. The target populations were persons with Turkish, Polish, Romanian, Syrian and Croatian citizenship living in Germany (AAPOR RR1 15,9%; N: 1190) and focused only on the web-based interviews. We used logistic regression to determine factors associated with mobile device usage. Furthermore, we investigated potential device effects. Therefore we constructed different data quality indicators: missing values, straightlining in grid choice questions, potential social desirable items and survey duration.


Female respondents were more likely to participate via smartphones than male respondents. Higher educated participants were more likely to participate via desktop compared to participants with lower educational level. We found no significant differences comparing the overall item nonresponse rate between desktop and smartphones. Furthermore there are no differences in the level of reporting in sensitive items. Participants who responded with a smartphone have a significantly longer completion time of the survey than respondents who participated via desktop. Our investigation also showed a significant negative device effect for smartphones on straightlining for two variables.

Added Value:

Implementing a survey option optimized for mobile phones could lead to a higher completion rate in typical hard-to-reach-populations, e.g. the low educated. This implementation might not compromise the data quality, since there are just minor differences on data quality between desktop and smartphone participants.

Öztürk-The impact of a mobile option in a migration survey-279.pptx

Using kinship big network data to overcome mistrust in recruiting the hard-to-reach populations: the case of Formosan endangered language survey

Ji-Ping Lin

Academia Sinica, Taiwan

Relevance & Research Question: Hard-to-reach populations are those hard to access due to geographical location or/and social status. They are characterized by being vulnerable, excluded, and hidden in a society. Recruiting hard-to-reach populations has long been a big challenge for survey study. Barriers relevant to recruiting hard-to-reach populations are various. The most crucial challenges are: how to (1) label the population for study, and (2) overcome mistrust of participants during survey. Using Formosan endangered language survey for example, the research utilizes (1) household individual data to label hard-to-reach population and (2) a kinship network database to help overcome the mistrust during survey.

Methods & Data: The survey study aims to access language skills, including listening and speaking of Taiwan indigenous peoples (TIPs). TIPs are an ideal example of hard-to-reach population. The research uses household registration data of TIPs to help label potential hard-to-reach population for study and thus to facilitate sampling design and survey strategy. Since mistrust serves as the most important barrier during survey once a potential individual is labelled. To overcome mistrust issue, the research makes use of a complex kinship network database that is construed based on computational social science. The database enables us to identify a group of persons (parents, siblings, relatives, and friends) who are kin to the person for survey. If the person for survey refuses to participate due to mistrust, we turn to seek for assistance from those she/he may know well, in a hope to reduce mistrust. All measures and strategy mentioned are examined by an IRB board.

Results: Utilizing population information in household registration helps labeling potential hard-to-reach populations. But more important is that making use of constructed kinship network database substantially helps us contact those who may know well about the sampled individuals. Such measure in turn help to reduce the mistrust of sampled population and increase survey participation rate.

Added Value: the survey results of endangered Formosan languages shed lights on the determinants of language utilization and language shift. We thus are able to propose relevant policy suggestions.

Lin-Using kinship big network data to overcome mistrust-189.pptx

Linguistic Properties of Echo Chambers and Hate Groups on Reddit

Robert Luzsa, Susanne Mayr

Universität Passau, Germany

Relevance & Research Question:

There is a growing debate about Echo Chambers (EC), that is, online groups in which like-minded people share attitudinally consistent information and which might amplify societal polarization. First attempts to analyze the psychological underpinnings of EC have been made, but so far there is little evidence regarding differences in conversational style between EC and more neutral, attitudinally diverse groups. Based on psychological theories like Self-Categorization Theory, EC can be expected to exhibit a more polarizing conversational style, for example by containing more pronouns referring to in- and outgroup members. Moreover, EC share structural characteristics with Hate Groups (HG, online groups propagating hate and violence), for example their users' ideological homogeneity. Therefore, some linguistic characteristics previously found in HG, such as more negative emotion and swearing, should also occur in EC. We tested these assumptions by examining user-submitted texts from online groups of the website Reddit.

Methods & Data:

We analyzed 14.642 user-submissions and 2.230.802 user-comments from 14 groups: Six neutral groups (e.g. /r/neutralpolitics), six EC, that is, ideologically homogenous groups explicitly forbidding divergent opinions (e.g. /r/latestagecapitalism), and two HG already banned by Reddit due to inciting violence (/r/incels, /r/physicalremoval). Linguistic properties (word type percentages) were calculated via the program LIWC and compared between EC, HG and neutral groups via MANOVA.


Compared to neutral groups, EC as well as HG displayed a more polarizing style with significantly more pronouns referring to in- and outgroup members, more plural than singular references, and more negative and less positive emotion. Additionally, this style was more pronounced in HG, which also displayed significantly more swearing and "you"-references than EC.

Added Value:

The results demonstrate that systematic linguistic differences between neutral and problematic online groups exist. Such differences might be used to build classification models that help platform providers and moderators to identify online groups requiring intervention. To illustrate this, we trained two logistic regression models with elastic net regularization that classify user-submissions as EC vs. neutral and HG vs. neutral based on linguistic characteristics. Both performed well with correct classification rates of 88% and 76%.

Luzsa-Linguistic Properties of Echo Chambers and Hate Groups-273.pdf
2:15 - 3:30F 3: Poster Session (Part III)

Style for Success? A Study on the Impact of Avatars’ Styling on Perceived Competence and Warmth.

Katja Markewitz, Patricia Glinski, Marius Herold, Carolin Straßmann, Annika Arndt, Sabrina Eimler

Hochschule Ruhr West, Germany


Avatars representing humans in a virtual environment are used in different online scenarios. One future application might be a digital assessment center, where candidates got represented by an avatar to design an inclusive application process. Based on the media equation theory (Reeves & Nass, 1996), prior evidences showing that styling has an influence on the evaluation of women (Klatt, Eimler & Krämer, 2016) might also be applicable to avatars. Nevertheless, it is still questionable, if and how this evaluation impacts the perception of the represented candidate’s capabilities. Thus, this study investigates the influence of an avatar’s styling on its perception and whether it has an effect on the perceived leadership abilities of the represented human.


To examine this question we conducted an online experiment with a 2 x 2 x 2 (skirt/pants, loose hair/braid, with/without makeup) between-subjects design. To enhance the generalizability two different figures have been evaluated and collapsed for the analyses. Overall 143 participants (55 female, Mage = 30.31, SDage = 13.28) evaluated the virtual woman concerning warmth, competence, status and leadership abilities.


The results showed, that avatars with makeup were rated more competent (F(1,135) = 5,801, p = .026, η2 =.036), evoked a higher leadership ability (F(1,135) = 7,309, p = .008, η2 =.0.051 and a greater chance of getting hired (F(1,135) = 4,01 p = .047, η2 =.029) in comparison to no make-up. Additionally, avatars with a braid are perceived as more competent (F(1,135) = 6.578, p = .011, η2 = 0.41), are associated with higher leadership ability (F(1,135) = 7,274. p = .008, η2 = .051 ) and had greater chances to get the job (F(1,135) = 5,85, p = .017 η2 = .042) than ones with loss hair. Moreover, for avatars with loss hair no make-up leads to a higher warmth perception than make up (F(1,135) = 5,565, p = .020, η2 = .040). No differences for clothing occurred.


The results show that it is important to be careful while designing the look of a digital avatar, because styling has an effect on the perception of the avatar and this evokes differences in the perceived capabilities of the human represented by it.

Markewitz-Style for Success A Study on the Impact of Avatars’ Styling on Perceived Competence and Warmth-287.pdf

Fake News: On the Influence of Warnings and Personality

Tanja Messingschlager, Fabian Prietzel, Stefan Krause, Julia Winkler, Markus Appel

University of Würzburg, Germany

Relevance & Research Question:

Misinformation on social media, often called fake news, has moved to the center of public discussion. One way to combat the spreading of fake news is the implementation of warning messages (e.g., “contested by independent fact-checkers”) attached to social media posts. However, the effectiveness of warning labels is debated and empirical research in this field is rare. Based on prior theories in offline contexts we expected warning messages to decrease the perceived accuracy and the intention to share such posts.

Importantly, we assumed that the effect of warnings varies with users’ personality, the Dark Triad of personality (narcissism, Machiavellianism, and psychopathy) in particular. Individuals with high Dark Triad scores are low on empathy, they disregard others, and tend to disrespect justice and truth. Thus, these variables should predict higher accuracy ascribed to social media posts with fake news warning labels and higher sharing intention of misinformation.

Methods & Data:

An online experiment was conducted (N=438). Facebook posts with and without warning messages were prepared that highlighted a subsequent news article. After reading the news article the perceived accuracy and the intention to share the content were assessed. Narcissism, Machiavellianism, and psychopathy scores were obtained. This experiment was based on a one-factorial between-subjects design with Dark Triad scores as continuous moderators.


On average, the effect of warning messages on perceived accuracy and intention to share was small. As expected, the impact of warning messages decreased with participants Machiavellianism and Psychopathy. The more individuals are predisposed to disregard others and to disrespect justice and truth, the less warning messages affect their judgment and intended behavior.

Added Value:

Warning messages are an often-discussed tool to counter the spreading of fake news but little is known about their immediate psychological effects. This project addressed this research lacuna and showed that personality traits predict the handling of misinformation.

Looking back. Moving forward. 20 years of GOR.

Marie-Luise Nau, Florian Tress

Norstat Group, Germany

Relevance & Research Question:

After the 20th GOR conference took place in Cologne last year, we thought it was time to examine the evolvement of the event. Its beginnings date back to the very early days of online research and prevailed through a very dynamic era with rapidly changing technologies. We wanted to know, what topics appeared and disappeared over the course of time and how the focus of the research community may have shifted.

Methods & Data:

With a little help of DGOF’s managing director Birgit Bujard, we collected and consolidated all available abstracts from the past twenty events between 1997 and 2018. As the setup of the conference has changed over the last decades, the data needed to be formatted, cleansed and translated into English, in order to create a comparable data set. Based on this data, we conducted a (text) analysis and visualized our most interesting findings.


Our infographic shows that there are temporarily trending topics, but also topics, which seem deeply ingrained in the DNA of GOR and the online research community. We show the development of certain key topics and also other KPIs that illustrate the evolvement of the conference.

Added Value:

Our poster represents some of the identity of the General Online Research conference, but is also meant to surprise and hopefully entertain the conference participants.

Nau-Looking back Moving forward 20 years of GOR-266.pdf

Respondents behavior in web surveys: Comparing positioning effects of a scale on impulsive behavior

Cornelia E. Neuert

GESIS-Leibniz Institute for the Social Sciences, Germany

Relevance & Research Question:

Previous research has shown that the quality of data in surveys is affected by questionnaire length. With an increasing number of survey questions that respondents have to answer, they can become bored, tired and annoyed. This may increase respondents’ burden and decrease their motivation to provide meaningful answers which might lead to an increased risk of showing satisficing behavior.

Methods & Data:

This paper investigates effects of item positioning on data quality in two web surveys. The first is an eye-tracking study with 130 participants and the second an online survey with 900 respondents. In each study, respondents answered a grid question on impulsive behavior that consists of eight items and a five-point response scale. The scale was randomly provided either in the beginning or at the end of the web questionnaire.


The position of the scale was predicted to influence a variety of indicators of data quality and response behavior in both web surveys: item nonresponse, response times, response differentiation, as well as measures of attention and cognitive effort operationalized by fixation counts and fixation times (only available for the eye-tracking study). Results show that data quality is lower for questions positioned later in a questionnaire which is shown in less item differentiation, shorter response times, less fixation times and less fixation counts.

Added Value: This study adds to the existing research on the optimal positioning of variables in surveys.

Neuert-Respondents behavior in web surveys-267.pdf

Fit for Industry 4.0? – Results of an Empirical Study

Swetlana Franken, Lotte Prädikow, Miriam Vandieken, Malte Wattenberg

Bielefeld University of Applied Sciences, Germany

Relevance & Research Question: Digitization and IoT have become the drivers of a far-reaching transformation process in companies worldwide. Companies are now faced with the challenge of shaping this change while considering the people and the organisation, in addition to technology. The aim of this study was therefore to examine the effects of the digitization of company employment and competence requirements, differentiated according to employee groups.

Methods & Data: Following preliminary literature research and qualitative expert interviews [n=6], a research framework was developed that consists of two interconnected levels: [1] requirements of internal and external digitization as well as [2] qualifications and competencies of different occupation groups.

Based on this, a quantitative online-study was conducted from Oct. 2017 to Jan. 2018. Participants [n=150] were recruited using a personal approach and consisted of company representatives from Germany with expertise in the field of digitization and HR.

Results: Concerning internal focus, 73% of companies surveyed have an ERP system and 69% have an intranet, but only 36% use a cloud-system, 29% data analytics and 8% AR/VR.

Regarding external factors, 50% do not have an online shop or a platform for customer communication. 77%, however, confirm the examination of new digital business models. Additionally, most respondents do not expect any employment effects from digitization but do expect a change in tasks and a greater need for training, especially for skilled workers (85%, academics 84%, unskilled 66%).

Openness to change is regarded as the most important competency across the employee groups, followed by the ability to learn for unskilled (90%) and skilled (88%) workers, the ability to think in context for academics (97%) and communication skills for managers (96%). While the most important task for managers is the design of framework conditions, for other employee groups it will be working with new technologies and data analysis


Added Value: The results show the status quo and untapped potential of these companies. It is clear that, among IT and media skills, companies are faced with other qualification needs and new areas of responsibility within the scope of digital transformation, which differ according to occupation group.

Franken-Fit for Industry 40 – Results of an Empirical Study-289.pdf

How to catch an online survey cheater

Manuela Ravagnan, Marco Fornea

Demetra s.r.l., Italy

Relevance & Research Question:

Online surveys are self-administered by respondents seeking to receive incentives for completing questionnaires. Some respondents use minimal cognitive effort in order to quickly complete the survey and receive the incentives. However, this can trigger behavior such as not reading the questions carefully, racing through the survey or intentionally cheating, resulting in poor data quality. This paper aims to investigate the behavior of cheaters among online respondents from a non-probability-based panel analyzing seven techniques for detecting cheaters applied in different ways in order to find an efficient methodology that leads to the exclusion of the greatest possible number of cheaters without eliminating honest panelists.

Methods & Data:

We used data from 2 web surveys conducted in Italy (during January 2019) on members of our own panel,, which is composed of 20,558 active panelists. The 2 surveys considered in our study have the following common characteristics: a sample size of 1,000, population target and a food consumption topic. Sample members were stratified by geographic area, gender and age in order to be representative of the Italian population. In both questionnaires, we asked a particular question that we used as target variable. The techniques used to detect the cheaters are: direct instruction in the body of the question, straightlining checks, speeder checks, trap questions (fake brand/names), open-ended question checks, multiple unlikely events in screening questions and consistency checks. We used the first survey as training set to determine a method for identifying cheaters. In particular, we analyzed the estimates of the target variable in each check and in any combination thereof, in comparison with a "gold standard". Once the method was defined, we validated it using the second survey as test set.


Preliminary findings show that removing respondents who fail a single quality control question does not improve data quality. In our analysis, participants flagged for removal should fail at least 2 quality control measurements.

Added Value:

Our paper aims to expand knowledge of cheaters and techniques to identify them. The main value of this work is the number of quality controls tested.

Ravagnan-How to catch an online survey cheater-274.pdf

How Much Text Is Too Much? Assessing Respondent Attention to Instruction Texts Depending on Text Length

Tobias Rettig

University of Mannheim, Germany

Relevance & Research Question:

Whether respondents pay adequate attention to a questionnaire and the stimuli within it, has been a concern for survey researchers for decades. One way of assessing attention is asking respondents for specific answers or actions, known as an instructional manipulation check (IMC). Previous research into this field has largely dealt with the question whether respondents read texts or not, but not with how much text they can be expected to read. I fill this gap in the literature by including an IMC in an online panel survey and systematically varying the length of the surrounding text.

Methods and Data:

Data stems from the November 2018 wave of the German Internet Panel (GIP), an online panel representative of the German population. About halfway into the questionnaire, respondents are instructed not to answer a specific question, but to continue by clicking the GIP logo instead. This instruction was “hidden” in the question text, the length of which was experimentally varied between four conditions: (1) Only the instruction was displayed, (2) the instruction was placed in one paragraph of text, (3) the instruction was placed in the second of two paragraphs of text, and (4) the instruction was placed in the fourth of four paragraphs of text.


Whether respondents will carefully read a text strongly depends on its length. The passing rate for the IMC ranges from about 80% for the shortest to under 40% for the longest text condition. The more text respondents are asked to read, the fewer of them will actually do so. While lower attention from respondents using mobile devices is a commonly voiced concern, I find no evidence to support this.

Added Value:

Respondents were previously often treated as either attentive or not, yet my results show that whether respondents carefully read a text strongly depends on how much text they are asked to read. Respondent attention can therefore be optimized for by keeping stimuli short. The results also indicate that respondents using mobile devices do not pay less attention to the survey.

Rettig-How Much Text Is Too Much Assessing Respondent Attention-148.pdf

PC versus mobile survey modes: are people's life evaluations comparable?

Francesco Sarracino1, Cesare Antonio Fabio Riillo2, Malgorzata Mikucka3

1STATEC research, Luxembourg; 2STATEC research, Luxembourg; 3MZES, Mannheim University (Germany),

The literature on mixed mode surveys has longly investigated whether face-to-face, telephone, and online survey modes permit to collect reliable data. Much less is known about the potential bias associated to using different devices to answer online surveys.

We compare subjective well-being measures collected over the web via PC and mobiles to test whether the survey device affects people's evaluations of their well-being. We use unique, nationally representative data from Luxembourg which contains five measures of subjective well-being collected in 2017. The use of multinomial logit with Coarsened Exact Matching indicates that the survey tool affects life satisfaction scores. On a scale from 1 to 5, where higher scores stand for greater satisfaction, respondents using mobile phones are more likely to choose the highest well-being category, and less likely to choose the fourth category. We observe no statistical difference for what concerns the remaining three categories. We test the robustness of our findings using three alternative proxies of subjective well-being. Results indicate that survey tools do not induce any statistically significant difference in reported well-being. We discuss the potential consequences of our findings for statistical inference.

2:15 - 3:30F 4: Poster Session (Part IV)

Optimizing Response Rates in Web Surveys of Establishments: The Effects of Contact Mode

Joseph Sakshaug1,2, Basha Vicari1, Mick Couper3

1Institute for Employment Research; 2University of Mannheim; 3University of Michigan

Identifying strategies that maximize participation rates in population-based web surveys is of critical interest to survey researchers. While much of this interest has focused on surveys of persons and households, there is a growing interest in surveys of establishments. However, there is a lack of experimental evidence on strategies for optimizing participation rates in web surveys of establishments. To address this research gap, we conducted a contact mode experiment in which establishments selected to participate in a web survey were randomized to receive the survey invitation with login details and subsequent reminder using a fully crossed sequence of paper and e-mail contacts. We find that a paper invitation followed by a paper reminder achieves the highest response rate and smallest aggregate nonresponse bias across all-possible paper/e-mail contact sequences, but a close runner-up was the e-mail invitation and paper reminder sequence which achieved a similarly high response rate and low aggregate nonresponse bias at about half the per-respondent cost. Following up undeliverable e-mail invitations with supplementary paper contacts yielded further reductions in nonresponse bias and costs. Finally, for establishments without an available e-mail address, we show that enclosing an e-mail address request form with a prenotification letter is not effective from a response rate, nonresponse bias, and cost perspective.

Sakshaug-Optimizing Response Rates in Web Surveys of Establishments-270.pdf

Brand Relationship Quality on YouTube: The emergence and impact of strong between recipient and creator relationships

Lucas Scheller

Trimexa GmbH, Germany

YouTube, with 1.5 billion users in 2017 (Statista 2017), has become the second largest social media platform. Some YouTube creators have gathered large fan communities. To get a deeper insight into the fandom of YouTube viewers and their bond with the creators, the multidimensional construct of Brand Relationship Quality (BRQ) by Fournier (1994) was transferred to the recipient-creator relationship. Creators are understood as personal brands for this study. The study also reviews the impact of BRQ on the brand loyalty of viewers towards creators, in order to see if a high BRQ is able to keep a creator relevant in the long term.

Another goal of this study is to examine the extent of influence a creator might have concerning their BRQ. For this purpose, the influence of the creator related determinants of authenticity, expertise, self-disclosure and brand personality strength was measured. In order to examine the mentioned relationships, a corresponding model is developed.

An online survey was conducted and a total of 713 subjects took part in it. Respondents were viewers of one out of four different YouTube creators.

To measure the dimensions of the BRQ, existing scales of Leung (2016), Thorbjörnsens (2002) and Henkel/ Huber (2005) were adapted to the recipient-creator relationship.

To determine the relationships between the creator-related determinants, a multiple linear regression analysis was performed.

The impact of BRQ on brand loyalty was determined using a simple linear regression analysis.

It can be verified that the BRQ with a R of, 518 has a highly significant impact on brand loyalty.

The influence of the creator-related determinants is collectively model R-squared, 504 and is also highly significant. However, the influence of the determinants of authenticity and expertise could not be confirmed.

It can be confirmed that the BRQ has a positive influence on the loyalty of recipients. The BRQ thus helps to keep a creator relevant in the future.

In addition, a creator can have a positive impact on the BRQ by increasing its perceived self-disclosure and brand personality with their behavior.

Scheller-Brand Relationship Quality on YouTube-293.pptx

SurveyMaps: A sensor-based supplement to GPS in mobile web surveys

Stephan Schlosser1, Jan Karem Höhne2, Daniel Qureshi3

1University of Göttingen, Germany; 2University of Mannheim, Germany; 3University of Frankfurt, Germany

The use of mobile devices, such as smartphones, to participate in web surveys has increased tremendously in recent years. The reasons for this development are a skyrocketing proportion of smartphone owners accompanied by an increase in high-speed mobile Internet access. This development also enables respondents to participate in web surveys without any time and place restrictions. For instance, they can take part on their morning way to work or at home in the afternoon. There are almost no limitations regarding mobile web survey participation. One strategy to investigate respondents’ position and completion conditions is the collection of GPS (Global Positioning System) data. However, a drawback of GPS is that its reception in buildings or beneath the ground is reduced or even impossible. We therefore propose “SurveyMaps (SMaps),” a JavaScript-based tool that combines passive data, such as IP address, acceleration, and compass, to provide a supplement to GPS data. In this usability study, we initially investigate the proper functioning of SMaps in public spaces, such as parks and cities, by using GPS data. To illustrate the functionality of SMaps, we present the results of a study in which we ask respondents to complete a mobile web survey on their smartphone while being outdoor and randomly walk around (informed consent will be obtained). The results of the pretests look very promising and indicate that SMaps reliably gathers respondents outdoor position. A next step is to test the application of SMaps in buildings and beneath the ground.


Integrating web tracking and surveys to investigate selective exposure in news consumption

Sebastian Stier, Johannes Breuer, Pascal Siegers, Tobias Gummer, Arnim Bleier

GESIS - Leibniz Institute for the Social Sciences, Germany

Relevance & Research Question: Computer-mediated communication has become deeply ingrained in political life. People use digital technologies to get political information and news or directly follow and interact with political actors. The measurement and analysis of these activities pose considerable challenges for researchers since they are distributed across multiple channels and platforms, intertwined, and ephemeral. With a case study relating a direct measurement of news consumption in Germany to political predispositions and attitudes of people we demonstrate the added benefit of integrating survey data and digital trace data.

The advent of right-wing populist actors and growing mistrust of political elites has coincided with negative public attitudes towards the news media. Given these political predispositions, selective exposure theory predicts that supporters of parties sending anti-media cues and people with populist attitudes would choose their news sources more selectively or even actively avoid entire types of news.

Methods & Data: We use a novel data set that links web browsing histories from 1,261 German internet users to their responses in an online survey. That way, we can objectively measure people's online behavior while surveying them for sociodemographic and political variables.

Results: Our findings in various regression models indicate that party cues are strongly related to news consumption while overshadowing the role of populist attitudes. Supporters of the German party with the most outspoken anti-media stances, the AfD, expose themselves less to news than supporters of other parties. This pattern remains robust for hard news, soft and tabloid news, and is most pronounced in the case of public broadcasting news.

Added Value: Our project makes substantive contributions to the fields of selective exposure and political communication. Methodologically we make a contribution to synthesizing the two paradigms survey research and computational social science as they have the potential to compensate for their respective weaknesses when combined in a systematic way.

Benchmarks for E-Health Evaluations

Meinald T. Thielsch1, David M. Kahre1, Carolin Thielsch1, Gerrit Hirschfeld2

1University of Münster, Germany; 2Bielefeld University of Applied Sciences, Germany

Relevance & Research Question: Two in three Internet users regularly search for health information online. Content, usability, and aesthetics perceptions as well as trust towards a website provider are central dimensions of web users’ experiences. Each dimension is crucial for users’ acceptance, appreciation, revisit and recommendation of specific websites. For example, if users do not understand the content or distrust the website’s provider, they will seek another one. The aim of the present study is to provide aids in terms of benchmarks, helping to better understand use and adoption of health information presented online.

Methods & Data: We combined data of two studies, with a total of more than n=2.500 participants, evaluating m=33 health-related websites. The website pool was based on evaluations of seven experts (including two of the studies’ authors). Study 1 (n = 355, m = 3) used a within-subject design; study 2 (n > 2.200, m = 30) used a between-subject design. Each website was presented fully-functional and rated by 60 to 355 participants (Mean = 105) using established questionnaires: Web-CLIC (website content), PWU-G and UMUX-Lite (usability), VisAWI-S (aesthetics), and trusting belief scales of McKnight et al. (2002). Study 1 participants were recruited via the panel PsyWeb (, study 2 participants via a commercial panel. Websites were clustered in different categories such as “charity websites”, “educational and clinician websites”, “government websites” or “media websites”.

Results: Data collection was only recently finished, and thus we just started the data analyses. We aim to test for significant differences between categories on the different web user experience questionnaires. Resulting benchmarks in terms of mean scores for each category and measure will be presented at the GOR.

Added Value: All widely-used evaluation tools yield continuous scores, e.g. leading to a website usability score of 5.5. In themselves these scores are difficult to judge – benchmarks and cut points enable a meaningful interpretation of such individual scores. This is of particular importance for websites presenting health-related information. Thus, we hope that both, scientists evaluating existing e-health services as well as practitioners creating new ones, find these benchmarks useful.

Online recruiting methods from the perspective of job candidates

Dilara Erdal, Meinald T. Thielsch

University of Münster, Germany

Relevance and research question:

Because of the shortage of skilled workers, demographic changes and the strong technological progress, personnel recruitment has become more important over the past years (Chapman et al., 2005; Weitzel et al., 2015). Furthermore, there is an increasing use of Social Media and mobile devices. Thus, online recruiting methods expanded the possibilities to recruit employees and are used even more by companies (Thielsch et al., 2012; Seng, Fiesel, & Krol, 2012, Weitzel et al., 2015). In the present study different online recruiting methods were analysed with regard to use and rating from an applicant’s point of view.

Methods & Data:

Nine online recruiting methods (e.g., corporate websites, online-communities, job portals, …) were examined in a web-based study. The sample (N=1121) was recruited and surveyed by respondi AG. For each method the participants had to indicate if they „know, but did not use“, „used before“, „use now“ or „do not know“ the following method. In addition, they rated these methods on a scale from 1 („does strongly not appeal to me“) to 5 („strongly appeals to me“).


Job advertisements on corporate websites and online job portals are used by most of the participants (55%). These methods have also the highest ratings (M=3.68, SD=1.073; M=3.50, SD=1.084). This is followed by Online-Communities like Xing or LinkedIn with a user rate of 25.5%. Among the examined Social Media platforms, Facebook has the highest user rate (18.3%) in comparison to Instagram (8.6%) and Twitter (9.0%). These platforms are not known by most of the participants (52.7%; 52.4%) and also have the lowest ratings (M=2.15, SD=1.122; M=2.10, SD=1.091).

Added Value:

The study shows that some online recruiting methods, such as online job portals or corporate websites, are often used by people searching for a job. Further, Facebook and online communities such as Xing or LinkedIn are used by many people – yet, some Social Media platforms (e.g., Twitter or Instagram) are unfamiliar to most of the respondents. Thus, many but not all online recruiting methods are used and accepted by most potential candidates.

The Relevance of IT-Competencies in a Digitalized Work Environment

Carina Groth, Malte Wattenberg, Miriam Vandieken

Bielefeld University of Applied Sciences, Germany

Relevance & Research Question: Digitalization and new technologies have entirely changed job design and workforce tasks. Several studies reveal that the competency requirements have risen in general and especially within digital and media skills. Yet, our overall understanding of competencies and specific changes in certain sub-areas of digital skills, both today and in the future, remains unclear. Based on this gap in research, the following questions were proposed:

What is the general understanding of the term “competency”? How relevant is digital and media competency in departments outside of IT? Which technologies and drivers of digitalization are most relevant employee tasks today and in the future (i.e. 10 years)?

Methods & Data: A 12-question web survey of employed persons with various jobs [n=261] was conducted in Nov.-Dec. 2017. Participants were addressed by social media, personal approach and faculty email lists. They were asked to answer technology related questions on a 4-point rating scale [1=unimportant, 4=very important].

Results: Most respondents (23.4%) understood competencies as “The situation-related adapted acting through acquired qualifications.” Other understands followed: “One’s self-organised ability to act in an existing situation” (19.9%) and “Application and implementation of the acquired knowledge into a certain action” (19.2%). The “Skill to deal with difficult tasks” (6.1%), however, ranked among the lowest of responses.

Concerning the relevance of digital competencies in certain departments, respondents believed that marketing & sales (3.45 points on average) and research & development (3.40) are most important. Least critical are the legal department (2.50) and production (2.78).

Respondents considered e-learning (2.99 today/3.39 in future), cloud computing (2.93/3.36), new business models (2.91/3.22) and big data (2.81/3.33) to be the most important technologies and drivers of digitalization. Although 3D printing (2.11/2.67) and VR/AR (2.30/2.93) ranked last, they had the widest distance regarding the future and thus untapped potential.

Added Value: This study reveals which specific media literacy related skills will be relevant in the future. Additionally, the study not only contributes to our understanding of the perception of the term competencies, but also shows which company departments are most affected by digital and media skills.

Groth-The Relevance of IT-Competencies in a Digitalized Work Environment-290.pdf

A selection bias of Facebook respondents, which need to be taken into account

Daniela Wetzelhütter1, Dimitri Prandner2, Sebastian Martin1

1University of Applied Sciences, Upper Austria; 2Johannes Kepler University, Linz

Relevance & Research Question: Social media has become an integral part of our daily life. Thus, it comes as no surprise that it is playing an increasingly important role in survey research. Despite methodological as well as factual criticism, surveys conducted via social media are growing more popular, especially when it comes to "Facebook". Companies with an Facebook account are asking their „Facebook-user“ to complete surveys.

Since social media usage is part of media usage and thus tied to the perceived uses and gratifications, influencing motivations on why individuals participate and how this affects survey data should not be ignored. To the best of our knowledge, no study exists that focusses on the influence of user-specific characteristics on the Facebook-survey participation and outcome, in order to interpret results in the correct way. Therefore, the following question arises: How can one account for an assumable “Facebook-survey-selection-bias”, triggered by the participants’ characteristics?

Methods & Data: German and Austrian public utilities collaborated to collect data via Facebook. Eight German and six Austrian utilities helped in the empirical study by posting an invitation for the inquiry on their Facebook-accounts. They shared the link to the online survey in 2017. 258 Facebook-users participated in the study. The participants were grouped based on configural frequency analyses. Indicators used for this procedure were involvement (importance of user-comments and of encouragement to ask questions) as well as confidence about the surveys’ impact (probability of change). Finally, linear regressions were performed in order to examine the impact of the group assignment.

Results: Three different groups of Facebook-respondents were computed. Group A consisted of respondents who want active involvement in current issues on the company’s Facebook page and have positive expectations about the impact of the inquiry, B is the opposite and C is in between. The group-allocation is responsible for different outcomes – the more active and positive a respondent seems to be, the more positive their answer. Moreover, the conducted “substantial” regressions gain a higher power to explain, when the grouping variable is considered.

Added Value: The paper highlights the importance of the participants’ characteristics for interpreting Facebook-survey-results.

Wetzelhütter-A selection bias of Facebook respondents, which need-220.pdf
3:30 - 3:45Break
3:45 - 4:45A05: Mixing the Modes
Session Chair: Ines Schaurer, GESIS Leibniz Institute for the Social Sciences, Germany
Room Z28 

Online, Face-to-Face or Mixed-Mode? Findings from a Methodological Experiment in the GGP Context

Almut Schumann1, Detlev Lück1, Robert Naderi1, Martin Bujard1, Norbert Schneider1, Susana Cabaço2, Tom Emery2, Peter Lugtig3, Vera Toepoel3

1Federal Institute of Population Reserach (BiB), Germany; 2Netherlands Interdisciplinary Demographic Institute (NIDI), Netherlands; 3Utrecht University, Netherlands

Relevance & Research Question: For a large panel study, the Generations and Gender Survey, that in the past has been conducted in CAPI mode only, this experiment investigates the chances and risks of partly moving online for the next round of data-collection. It tests a sequential mixed-mode (push-to-web) design, combining CAWI and CAPI modes. The particular circumstances to consider are a rather long questionnaire of approximately 60 minutes, a complex routing and a focus on family-demographic issues. We are comparing response rates, costs, representativity, accuracy of measurement, and further aspects of data-quality.

Methods & Data: The experiment has been carried out 2018 in three GGS countries – Germany, Croatia, and Portugal – with more than 1,000 respondents in each country. (Fieldwork is on-going on the day of submission.) In all three countries a reference group was interviewed in CAPI mode, while an experimental group was interviewed in a sequential mixed-mode design (CAWI and CAPI). In each country a further specific experiment was carried out: In Germany strategies of incentivation, in Croatia the timing of reminders, and in Portugal two modes of selecting a contact person within the household were compared.

Results: Since fieldwork is on-going only preliminary results are known so far: Regarding response rates and accurate measurement, the online mode works quite well, despite the interview length. This is particularly true with a generous incentivation strategy and closely timed reminders. However, break-off rates are high. And the consent to store contact information for a re-contact is clearly lower than in CAPI mode. The country context makes a strong difference. Largest problems were identified in Portugal, starting already with low contact rates. In the sequential mixed-mode, the CAPI follow-up has low response rates and tends to proceed slowly. It seems that fieldwork institutes are underestimating the number of interviewers needed.

Added Value: The experiment identifies recommendable design characteristics for a push-to-web design for the GGS, as well as for comparable panel studies. It provides comparisons of various aspects of data-quality between CAWI, CAPI and sequential mixed-mode. And it provides evidence for the importance of the country-context in that respect.

Schumann-Online, Face-to-Face or Mixed-Mode Findings from a Methodological Experiment-160.pdf

Design and Implementation of a Mixed Mode Time Use Diary in the Age 14 Survey of the Millennium Cohort Study

Emily Gilbert, Lisa Calderwood

University College London, United Kingdom

Relevance & Research Question: Time diary data provide a comprehensive and sequential account of daily life and are used for a wide range of analytic purposes. Recent years have witnessed a steady growth of large-scale time diary data collection in cross-sectional as well as longitudinal surveys, driven by the increased research interest in population activity patterns and their relationship with long-term outcomes. The majority of social surveys collect paper-administered diaries, which have been shown to produce the most accurate and reliable daily activity estimates, but present challenges relating to respondent burden and administration costs. The use of new technologies for data collection could address these weaknesses by providing less burdensome diary instruments, improving data quality, and simplifying post-fieldwork data coding costs.

Methods & Data: The Millennium Cohort Study (MCS) was the first large-scale longitudinal survey to use a mixed-mode approach for the collection of time use data among teenagers. A smartphone app, web diary, and paper diary were specifically designed for the sixth wave of the survey, when cohort members were aged 14. The smartphone app in particular was a departure from the more traditional methods of time use data collection. This presentation will focus on the development of the time-use instruments, including their design, development and implementation in the field, as well as take-up and selection into different time diary modes, data quality across the instruments, and mode differences in measurement.

Results: The app proved the most popular choice among 14 year olds, with 64% choosing to use it over the other instruments. Those who completed the app diary were more likely to be female, high users of social media, and those whose families had higher incomes. In terms of data quality, the app and web diary proved comparable, and both outperformed the paper diary. Differences in measurement, specifically when looking at reported time spent doing different activities, was seen across the three modes, with the app and web being more similar than the paper diary.

Added Value: The experiences of MCS provide practitioners with useful evidence for designing high-quality instruments to collect time use data online.

Gilbert-Design and Implementation of a Mixed Mode Time Use Diary-159.pdf

Understanding mode switching and non-response patterns

Alexandru Cernat

University of Manchester, United Kingdom

The shift from single mode designs to mixed mode, where a combination of interview modes are used to collect data, is one of the key challenges of contemporary longitudinal studies. The decision about what modes to use and how to implement them can have a big impact, influencing costs, field-work procedures, non-response and measurement error.

The research proposed here aims to better understand one of the key characteristics of a mixed mode design: how people transition from one mode to another in time. This is essential for a number of reasons. Firstly, it can inform targeting strategies. For example, it can be used to target those people that are more likely to shift from a cheap mode to a more expensive one. It can also be used in models for dealing with non-response after data collection, such as weighting models. Thirdly, it can be used to explain measurement error that appears due to the mode design.

This paper will investigate the process of changing modes by looking at waves 5-10 of the UKHLS Innovation Panel where a sequential Web-Face to face design was used. Latent class analysis will be used to find the underlying patterns of change in time of modes. The clusters found will be used both as dependent variables, to understand who are the types of respondents in each, and as independent variables, to predict future wave non-response and mode selection. Findings will inform the design and use of the main UKHLS study.

3:45 - 4:45A15: Online Survey Experiments
Session Chair: Diana Steger, Universität Ulm, Germany
Room 154 

On the Transportability of Experimental Results

Felix Bader

University of Mannheim, Germany

Relevance & Research Question: Online experiments are a promising complement to traditional methods of data collection as they allow, better than surveys, to collect real behavior and, better than laboratory experiments, to reach very diverse samples. Yet, it is unsure if online and lab experiments yield equivalent results and if it is at all necessary to broaden the sample, i.e. if incentivized decisions in typical situations depend on mode, sample, and setting of experimentation.

Methods & Data: We test for a mode effect by comparing prosocial behavior in typical interactive games using thoroughly parallelized lab and online sessions within student subject pools in Leipzig and Munich. We than broaden our online sample applying the same decision interface to the general population of Germany using Forsa’s offline recruited online access panel and a convenience sample in the United States and India using Amazon’s Mechanical Turk.

Results: It turns out that the mode of data collection has no systematic effect. Behavior in lab and online experiments is very similar within both cities, but there are notable systematic differences in students’ prosocial behavior between Leipzig and Munich. Students in Leipzig reveal stronger other-regarding preferences, but the general population is even more benevolent, and participants on Mechanical Turk behave even less prosocial than students in Munich.

Added Value: Behavior, also in incentivized decision situations, strongly depends on the sample under study. This contradicts the common attempt to generalize results from empirically motivated laboratory studies to establish descriptive parameters of human behavior and asks for replications of experimental results with broader samples. Like surveys, experiments can only tell about behavior of the population sampled from. Fortunately, results of online experiments are in line with lab experiments. Therefore, we have a method at hand to examine real behavior of non-student participants.

Bader-On the Transportability of Experimental Results-209.pdf

Price setting in a VUCA world: a simple approach to re-interprete the van-Westendorp-approach (PSM)

Andreas Krämer

University of Applied Sciences Europe, Germany

Relevance & Research Question:

While in the scientific literature in particular market tests, auctions and indirect methods such as conjoint measurement are preferred as a suitable method for determining price-demand functions (Breidert, Hahsler & Reutterer 2006), the direct query of willingness to pay (WTP) is often characterized by a low validity. However, these direct methods are frequently used in practice (Steiner & Hendus 2012). One of the most established - but at the same time - controversial approaches is the Price Sensitivity Meter (PSM) developed by Van-Westendorp (1976) with further extensions (Roll 2010)

Methods & Data:

As part of a multi-stage research approach, the first step was to examine how well the optimum price point (OPP) derived from the classic PSM meets the actual market price (Miller et al. 2011). This is based on 10 price optimization studies (online, n = 276 to 5,000, Germany). In a second step, PSM price points are re-interpreted. The two upper price points ("expensive but acceptable" and "so expensive that I would not buy anymore") are used for estimating the individual willingness to pay (WTP). Thereafter, revenue and profit-maximizing prices are derived. In the third step (experimental design), the suggested modified PSM approach is compared with other direct methods (including incentive-compatible procedures).


Results for first step show that the OPP derived from the classic PSM is problematic for price setting: Typically, recommended prices are lower than the market price. When using the modified PSM approach, optimal prices levels can be derived that are significantly closer to the market price. Advantageously, no additional data is necessary to validate PSM results, since price points are only interpreted differently. Both, the modified PSM approach and the direct open WTP-queries lead to similar recommendations for price setting.

Added Value:

Due to a changed market environment (VUCA, volatility, uncertainty, complexity and ambiguity) there is an increasing demand for simple and short questions, which can be used to determine the consumers´ WTP as well as optimal price levels. The proposed method can overcome limitations of the original PSM approach: it meets the needs of simplicity, speed, agility and research economics.

Krämer-Price setting in a VUCA world-116.pdf
3:45 - 4:45B05: Images and Virtual Reality in Market Research
Session Chair: Ruben Bach, University of Mannheim, Germany
Room 158 

If I can virtually touch it, I’ll buy it? Analysing the influence of (non) interactive product presentations in the online-grocery sector

Melanie Bender, Christian Bosau

Rheinische Fachhochschule Köln, Germany

Relevance & Research Question:

The online-grocery retail has significantly risen in recent years, still many consumers criticise the lack of testability of food (BDVM, 2018). “Need for Touch” (NFT), the desire to touch products before purchase (Peck & Childers, 2003), can be seen as one reason to refuse to buy food online (Lichters, Kühn & Sarstedt, 2016). But how can food be made more tangible? Current studies show that presentations such as “embodiment” (photo of holding food with hand) or “360-degree-rotations” lead to better product evaluations compared to photos (e.g. Elder & Krishna, 2012; Choi & Taylor, 2014). However, none of these studies have experimentally tested these presentations as a surrogate for a high level of NFT in an online-grocery-shop setting.

Methods & Data:

A mixed experimental 3x2x2 online-design has been used (rated by a representative sample of the REWE-Payback-Online-Panel (N=500)), which compared presentation (photo, embodiment, 360-degree-rotation), product (apple, noodles; repeated-measurement) as well as NFT (high, low) on six variables of product evaluation (e.g. involvement & purchase intention) considering perspective-takeover as covariate. To gain more detailed data regarding activation, the autorotated 360-degree-rotation has been divided into self-rotation and no-self-rotation based on paradata and self-disclosure.


The results of repeated-measures ANCOVA show that there are significant main effects for all dependent variables for presentation across both products (p ≤0.001; f= 0.17-0.43): The 360-degree-rotation is the best presentation form followed by the photo, but contrary to expectations embodiment scores worst. In addition, self-rotation leads to an even significantly higher purchase intention, while no-self-rotation is comparable to the photo (p ≤0.001; f= 0.25). Moreover, interaction analysis tendentially reveals that if consumers rotate the noodles by themselves, there is no difference in level of purchase intention between those who have a high or low need to touch products; therefore, self-rotation seems to act as a surrogate for high NFT (p ≥0.18; f= 0.01).

Added Value:

Overall this study emphasizes the importance of an activating shopping experience in terms of online-grocery. Consumers desire more vivid and tangible product presentations like 360-degree-rotations of food to be more ensured in online purchase decisions using their virtual touch.

Bender-If I can virtually touch it, I’ll buy it Analysing the influence of-128.pdf

Mobile Detection of Visual Brand Touchpoints

René Schallner, Carolin Kaiser

NIM, Germany

Relevance & Research Question

Consumers encounter contacts with brands frequently in everyday life: when opening the refrigerator, passing billboards on a tram ride, viewing ads in print/TV/online media, etc. Measuring such brand touchpoints can both be a valuable feedback channel for marketing and a great source for market research. However, measuring all touchpoints of a single consumer requires many sources and often only delivers incomplete or indirect data. We address the question of how visual brand touchpoints can be measured with a single-source approach and we compare our results to a traditional questionnaire.

Methods & Data

In this research, we present an approach to capture brand, time, duration, and location of brand touchpoints in real-time by applying computer vision methods on a low-cost mobile hardware prototype. We use a deep convolutional neural network for real-time logo detection on a smartphone that is capturing images from a USB webcam mounted on the frame of a pair of sunglasses. A mobile app collects the detected brand logos, time, and location for further analysis. To guarantee privacy, no images are stored; only textual results are saved. We apply this approach in a case study, where we collect data from 26 participants walking on a reference route with 17 known potential touchpoints for 5 brands identified by 5 logos and 2 letterings. We then survey the participants with a questionnaire, asking for a protocol of their remembered brand touchpoints, for the selected brands.


The performance evaluation of the mobile logo detection shows that 94% of logos are detected. In the case study, participants recall only 64% of all detected touchpoints and the touchpoint sequence from the questionnaire only overlaps with the detected one by 39%.

Added Value

Single-source detection of visual brand touchpoints provides valuable data for determining the influence and effectiveness of marketing measures. The case study shows that our method yields more reliable and more complete data than questionnaires. As additional benefits, our approach also captures the exact time, duration, and location of the touchpoints, thus revealing more detailed insights into consumers’ encounters with brands.

Schallner-Mobile Detection of Visual Brand Touchpoints-215.pptx

Revealing consumer-brand-interactions from social media pictures - a case study from the fast-moving consumer goods industry

Carolin Kaiser, René Schallner, Vladimir Manewitsch

NIM, Germany

Relevance & Research Question:

A multitude of pictures is posted on social media every day, shedding light not only on consumers’ social life but also their interactions with brands like holding, drinking or even hugging a soda bottle. These pictures represent a valuable source of knowledge for marketing which is infeasible to explore manually. This research addresses the question of how consumer-brand-interactions can be recognized and characterized automatically.

Methods & Data:

We present an approach to reveal types of consumer-brand-interactions from social media images by combining methods from computer vision and statistics. First, an image captioning method based on convolutional neural networks and recurrent neural networks estimates the polarity, involvement, and purpose of the consumer-brand-interaction and describes the consumer-brand-interaction in natural language. Afterwards, a clustering algorithm groups the images by polarity, involvement, and purpose and characterizes the different clusters by the subjects, predicates, and objects of the sentences describing the consumer-brand-interactions.

We apply this approach in a case study from the market of fast moving consumer goods (FMCG). The dataset comprises approx. 950.000 public user-generated images which have been posted on social media during a period of 5 years and which are related to 26 popular FMCG brands.


The evaluation of the image captioning approach yields good performance. Polarity, involvement, and purpose of consumer-brand-interactions are on average estimated correctly in 72% of the images and a correct sentence regarding the subject, predicate, and object are generated in 70% of the images.

The cluster analysis reveals 6 different types of consumer-brand-interactions for fast moving consumer goods ranging from random encounter, pre-usage, active usage, happy moment, and emotional engagement to endorsement.

Added Value:

In contrast to existing visual analytics approaches not only static objects such as products or people but dynamic interactions between consumers and brands are discovered. For example, it is not only possible to detect a woman and a soda bottle, but to differentiate whether she is sitting next to it or kissing it. Thus, marketers are in a better position to estimate the brand popularity and to tailor marketing campaigns or products to real live usage scenarios.

Kaiser-Revealing consumer-brand-interactions from social media pictures-206.pdf
3:45 - 4:45C05: GOR Thesis Award 2019 Competition: Bachelor/Master
Session Chair: Meinald Thielsch, University of Münster & DGOF, Germany
Room 149 

Interactions on Twitter conducted at a cMOOC – Results of a mixed-methods study

Jasmin Lehmann

Technische Universität Ilmenau, Germany

Relevance & Research Question. A massive open online course (MOOC) is an online course with open access and a theoretically unlimited number of participants. Since 2011, the potential of MOOCs and the associated problems have been discussed intensively. The public interest and the rapid dissemination of the courses underline their significance. Since the first MOOC in 2008, two different formats have been emerged: the behaviorist oriented xMOOC (extended MOOC) and the connectivist oriented cMOOC (connectivist MOOC).

This study deals with cMOOCs because - unlike xMOOCs - the format has a higher didactic potential. cMOOCs focus on shared, network based learning as well as on personal responsibility, motivation and commitment of the learners. Social media channels are integrated for sharing and networking. The interaction of the participants is considered to be a major factor for the success of a cMOOC. Nevertheless, there are few research results available in this area.

The aim of this explorative study is to contribute to the elucidation of the interactions within the cMOOC „Corporate Learning 2025 MOOCathon“. The investigated platform is Twitter. The social, medial and scientific relevance of Twitter can be explained by the increasing importance of the microblogging service. Twitter is a global phenomenon that is steadily growing in terms of users, news, and thus in usable data. The importance of the short message service as a learning tool was confirmed by an international survey of experts, in which Twitter ranked first among the 200 best learning tools for seven years (2009–2015).

Against this background, the following research questions emerged:

Q1: What types of interaction occur on Twitter during a cMOOC?

Q2: What interactions occur at what stage of the learning process on Twitter during a cMOOC?

Q3: How are the interactions within the learning process on Twitter perceived by the participants and organizers of a cMOOC?

Methods & Data. Moore’s three types of interaction and Salmon’s five-stage model of teaching and learning online were applied as a theoretical framework. Moore differentiates between three types of interaction: “learner-learner”, “learner-instructor”, and “learner-content” interaction. The first two types describe the interactions between learners or between learners and teachers. The interaction between learner and content refers to the individual’s interaction with the content.

The second model of the framework is the five-stage model of teaching and learning online according to Salmon. The basic idea is that the learners go through an individual learning process, which is divided into five stages: The first level is the stage of access and motivation and is about setting up processes and motivating learners to participate. The second stage is the stage of online socialization and is characterized by the formation of a community. In the third stage, the stage of information exchange, a common understanding among the participants is built up. This is the basis for the fourth stage, the stage of knowledge construction. In the fifth stage of development, learners reflect the acquired knowledge.

In a mixed-methods study it was analyzed which type of interaction occurred during the process of learning and how it was perceived by learners and organizers of a cMOOC. For that purpose n=2,497 Twitter messages of a partial sample were explored by a quantitative content analysis in 2018.

Based on the findings gained from the content analysis, an interview guideline was developed. Eight telephone interviews with two organizers and six learners were conducted. The sample of the learners was chosen considering the characteristics gender and activity level. The activity level was derived from the content analysis. Three male and three female persons were interviewed, the male and the female group each consisted of one person with a low, one with a medium and one with a high level of activity.

Results. The analyzed Twitter messages were published by 170 different users. 88% of the Twitter users were identified as learners, 8% as weekly hosts and 4% as organizers. 76% of the messages were published by the learners, 21% by the organizers and 4% by the weekly hosts (n=2,497).

The findings revealed that the most interaction occurred between learners on Twitter during the cMOOC. In the first stage of the learning process, the interaction between learners and organizers was the highest. The relevance of “learner-instructor” interaction at the beginning of the course was affirmed by this and other studies. In the second, third and fourth stage, interaction between the learners was most common. In the fifth stage, mainly the “learner-content” interaction took place. Twitter was used primarily for the second and third stage of the learning process – for online socialization and information exchange. The short message service was hardly used for the discussion and reflection of the content, which corresponds to the fourth and fifth stage of the learning process.

Overall, the participants interacted relatively often in their Twitter messages. Unlike in other MOOCs, the interactions – especially the interaction between the learners – did not decrease continuously over time. The study shows that the organizers of the cMOOC interacted regularly and very often with the participants. Furthermore, the presence of weekly hosts had no effect on the quantity of learners’ interactions.

Added Value. As the explorative study focuses on a particular case, the results can’t be generalized into being applicable to all cMOOCs, but the study provides a unique insight into an isolated instance of interactions within the learning processes on Twitter. The finding that can be of general value is the knowledge on cMOOC learner’s behavior and interactions on Twitter. MOOC designers and facilitators can consider these results in the conception and moderation of future MOOCs and thus can better engage with their learners.

Can these stars lie? Online reviews as a basis for measuring customer satisfaction

Nadja Sigle

Hochschule für Technik Stuttgart, Germany

Relevance & Research Questions:

Consumers who shop online find great value in the opinions and recommendations of other consumers when searching for product-related information. Online reviews written by customers in online shops or review sites therefore influence readers and consequently the success of products or services. Online reviews can be interpreted as the behavioral result of the experienced level of customer satisfaction. Assessing customer satisfaction is of major interest for companies, which is why product managers and other executives rely on various sources and measurements to gain insights from their customers. One method - that has been developed in recent years to gain more insight into product-related customer satisfaction - is the mining and monitoring of online reviews using automated software solutions.

However, studies have shown that these reviews might be compromised by biases affecting the customers writing them. Customers who purchased a product tend to have an overall positive attitude towards this product because they consciously decided to buy it, resulting in a more positive rating (purchasing-bias; Hu, Pavlou & Zhang (2007)). Furthermore, the distributions of star ratings, the most common way of rating satisfaction with a product on platforms like Amazon, are not normally distributed. Instead they often follow a “J-shaped” distribution which means that online reviewers rate products rather extremely than moderately. Research suggests that this is the result of a self-selection of customers moderately satisfied with the product. Moderately satisfied consumers lack motivation, like excitement or disappointment, to write an online review (under-reporting bias; Hu et al. (2007)).

The described shortcomings of online reviews lead to the question whether they are an appropriate source for companies to derive customer satisfaction from. To approach this issue in the current study, the method of review monitoring was compared to the method of customer satisfaction surveys, a widely established method of measuring customer satisfaction. The study followed two research questions:

1. Can online reviews give a realistic image of the product-related customer satisfaction?

2. What is the added value of review monitoring compared to online surveys and how can the two methods complement each other?

Methods & Data:

Washing machines were chosen as research subject to compare the two methods of satisfaction measurement, an object that is available in any household and is usually bought after an extensive search for information. The data basis was limited to six washing machine brands in order to increase the comparability of the data.

The study followed a four-step research process. First, a code system was developed in order to categorize all relevant evaluation criteria for washing machines. The data basis were 300 online reviews for washing machines. Eight superordinate performance parameters could be identified in the review texts, e.g. washing, price and customer service. The overall satisfaction assessment corresponds to the five-star scale used in all reviews.

Second, a questionnaire based on the code system developed in the first step was designed in order to measure the product-related customer satisfaction. The questionnaire contained open-ended questions as well as five-level scales to assess the satisfaction with the washing machine in general and its performance parameters. Additionally, the relevance of assessment criteria was measured. The data was collected via an online survey. The sample consisted of 510 washing machine users who purchased a washing machine between 2016 and 2018 and it was composed of 45% women and 55% men with an average age of 45 (SD = 12.4).

Third, a sample of 589 consumer reviews was drawn and coded with the previously developed code system. The sample was composed of 57% women, 36% men and of 7% authors with unknown gender. The open-ended questions from the online survey were coded with the help of the same code system.

In the last step, the data retrieved from the user reviews and from the satisfaction survey were analyzed and compared on a quantitative and qualitative level.


Results of the analysis of the overall satisfaction suggest that authors of online reviews are slightly more satisfied with their washing machines than the survey participants. But the differences in ratings are only very small. Both the distributions of ratings from online reviews and the survey are “J-shaped”, meaning they are not normally distributed but rather polarized. However, online reviewers rate their washing machines more often with extreme values on the satisfaction scale than the participants of the survey. These findings support the existing research on the under-reporting bias.

Analyses of assessment criteria and performance parameters show that authors of online reviews mention more distinctive criteria in their review texts than participants of the surveys do in open-ended questions. In addition, authors of online reviews mention criteria that are perceived as important more often than the participants of the survey. This indicates the additional qualitative value of online reviews.

The qualitative analysis shows that online reviews can provide a genuine insight into the usage of the appliances and into the living environments of customers. The distinct performance parameters are rated similarly by authors of online reviews and participants of the survey, which suggests a consistency in more detailed assessments.

Added Value:

The study shows the advantages and disadvantages of both methods to measure customer satisfaction. Results show that online reviews are indeed biased to some extent. The distribution lacks moderate ratings that are more common in customer satisfaction surveys. This can lead companies to over- or underestimate the customer satisfaction with their products when only referring to online reviews. On average, products are rated very similarly with both methods, which speaks for their reliability. In terms of qualitative insights into the use of products or product features that can disappoint or excite customers, online reviews can be a valuable source for companies e.g. in product management or development.

Overall, the method of review monitoring appears to be a valuable tool for companies to gain a real-time overview of customer feedback and satisfaction with their product portfolio. Especially for companies that offer various product categories with different models, it is a way to track problems systematically and quickly in a cost-effective and continuous manner. It is recommended to use additional market research methods, such as customer satisfaction surveys, to balance biases and to validate the qualitative input gained from online reviews with more quantitative measures. The mix of methods can provide a comprehensive view of product-related customer satisfaction.

Comparing the Portrayal of German Politicians in Bing News and Google News Search Results

Marius Becker

Technische Universität Ilmenau, Germany

Relevance & Research Question:

The internet is crucial for keeping up with news: Almost 60 percent of the German population above 14 years access online news (van Eimeren & Koch, 2016). The majority of German internet users rely on (news) search engines to navigate online information (van Eimeren & Koch, 2016). By filtering the available (news) content and deciding which content to present to the users, these news aggregators act as secondary gatekeepers (Nielsen, 2014; Singer, 2014; Wolling, 2005). Editorial lines and selection criteria of (human) primary gatekeepers are generally known, but there is a lack of information about the workings of news search engines’ selection and ranking algorithms (Lewandowski, 2015).

The findings of several US-American studies indicate that there may be relevant differences between different news search engines. These observed differences point to two dimensions: the variety of presented news sources , and the portrayal of politicians on a content level. Compared to Yahoo News, the search results of Google News showed a larger variety of sources and less concentration on big publishing companies (Bui, 2010). Search results for the terms “George W. Bush” and “John Kerry” retrieved by Yahoo News were more neutral, while Google News presented more explicitly judgmental news articles (Ulken, 2005). Thus, by relying on these services users might – without being aware of this – receive differing information about the same subjects (Bozdag & van den Hoven, 2015).

However, there are considerable research gaps - especially in the context of specialized news search engines. First, there is a lack of comparisons between different news search engines in the literature. Most studies focus on only one service or compare search engines and other services (e.g. news portals). Second, only very few studies compare the search results on a content level. Third, the few comparative studies that consider the retrieved content are all rather old and focus on the US. These past findings may no longer apply and are not necessarily valid for the German editions of the news search engines.

This study aims to close these research gaps. In addition to a descriptive examination of the retrieved sources, the study focuses on the portrayal of politicians in two popular news search engines in Germany. Two dimensions of portrayal are considered for this comparison: First, the portrayal of politicians’ private lives (privatization), which includes information on lifestyle, families, and friends (Holtz-Bacha, Langer & Merkle, 2014). Second, the portrayal of politicians’ professional characteristics as leaders (leadership images), such as political skills, vigorousness, and charisma (Aaldering & Vliegenthart, 2016).

The study is guided by the following research questions:

To what extend does the portrayal of German politicians differ between Bing News and Google News results?

SRQ1 Are the two news search engines’ results favoring certain politicians by portraying them in a better light than others?

SRQ2 Are the two new search engines’ results favoring certain political parties by portraying their politicians in a better light than others?

Additionally, this study tries to identify potential differences in the types of portrayals themselves:

SRQ3 Are the two news search engines focusing on professional or private information about the politicians?

SRQ4 Are the two news search engines emphasizing different dimensions of leadership in their portrayal of politicians?

Method & Data:

The empirical assessment of search engine algorithms is challenging, as researchers can only compare outputs for identical inputs to infer potential differences between services (Lewandowski, 2015). Thus, this empirical examination followed a three-step process. First, 20 search queries with 16 different search terms (names of politicians and political parties, current political issues) were sent simultaneously to the selected news search engines between December 2017 and January 2018. The first search result page was archived for each query and the first ten entries were considered for analysis. Second, the first five news articles per result page were accessed and archived. To avoid unwanted personalization based on the accessed news sources, the first step was fully completed before starting the second. Third, the resulting dataset of 400 search results and 200 full-length news articles was examined via quantitative content analysis. The analysis focused on the variety of presented sources and the portrayal of the three first-mentioned politicians per article. The portrayals were coded as positive, negative, or neutral. In total, 20 categories on the portrayals of politicians – 8 for privatization (Holtz-Bacha et al., 2014) and 12 for leadership images (Aaldering & Vliegenthart, 2016) - were considered.

Results (excerpts):

SRQ1: There are no signs of a systematic bias for or against specific politicians. However, there is a lot of variance in the portrayals and there are two individual cases in which the portrayals of the same politicians differ significantly between the search engines. Additionally, the portrayals in Google News results are more likely to be negative (42%) or positive (28%) than portrayals in Bing News results (negative: 37%, positive: 17%, χ² = 9.821, p = .007).

SRQ2: There are no indicators for systematic bias for specific political parties with regard to the portrayal of associated politicians.

SRQ3: Regardless of the selected search engine, information about politicians’ private lives play almost no role in the observed portrayals of politicians.

SRQ4: There are no clear indicators of the search engines emphasizing different dimensions of leadership.

Additionally, there are significant differences in the selected news sources: Bing News results contained more online-only media sources (25%, Google: 8%, χ² = 23.91; p < .001) and wire reports (52%, Google: 27%, χ² = 13.10; p < .001). Google News retrieves more broadcast sources (16%, Bing: 9%, χ² = 23.91; p < .001) and – at least in this sample – more opinion articles (12%, Bing: 6%, χ² = 2.20; p =.138).

Added value:

Although no systematic bias considering the portrayal of politicians is evident, the choice of (news) search engine influences the content that will be presented to the users. The findings indicate that algorithmic bias might manifest in different dimensions than expected. Instead of a political left-right bias, the significant differences in the presented sources and in the portrayals of some politicians show that new forms of bias should be considered in future research.

Citizens also need to be educated about secondary gatekeepers as part of code literacy: The commonly cited advice to cross-reference information between different sources might have to be expanded to also include cross-referencing between different services to find these sources of information.

3:45 - 4:45D05: Stichprobenqualität und Repräsentativität in der Online-Forschung
Session Chair: Bernad Batinic, JKU Linz, Austria
Session Chair: Horst Müller-Peters,, Germany
Room 248 

Warum gute Online-Forschung nur mit guten Stichproben möglich ist

Thorsten Thierhoff

forsa GmbH, Germany

Es geht um nicht weniger als den elementaren Anspruch der quantifizierenden Sozialforschung: Erkenntnisse, die in einer relativ kleinen Stichprobe gewonnen werden, sollen für eine wesentlich größere Gruppe, die sogenannte Grundgesamtheit, aussagekräftig sein. Doch kein Thema in der Online-Forschung ist so heftig umstritten, wie die Frage der „Repräsentativität“ von ausschließlich online durchgeführten Umfragen. Bei allen technischen Vorteilen, die der Siegeszug des Internets zweifelsohne auch für die Umfrageforschung mit sich bringt, so hat die Online-Welt ausgerechnet bei der so entscheidenden Frage der Stichprobenbildung innerhalb des eigenen Mediums keine zufriedenstellende Lösung im Angebot. Dabei ist der Weg, auch unter dem Gesichtspunkt der Repräsentativität, „gute“ Online-Forschung durchzuführen gar nicht sonderlich kompliziert: Benötigt werden jedoch qualitativ hochwertige Stichproben. Doch während es für Face-to-Face- oder Telefonbefragungen funktionierende und von einem breiten Forscherkreis allgemein anerkannte Auswahlgrundlagen gibt, fehlt eine vergleichbare Basis in der Online-Welt. Gute Stichproben für Online-Befragungen müssen also weiterhin ihren Ursprung in der Offline-Welt haben. Entsprechende Auswahlgrundlagen sind, beispielsweise im forsa.omninet-Panel, verfügbar und liefern schnelle und zuverlässige Ergebnisse. Ein Plädoyer für mehr Qualitätsbewusstsein in der Online-Forschung.

Gut gewichtet ist repräsentativ genug? – Ergebnisse einer Eigenstudie zur schwedischen Parlamentswahl 2018

Florian Tress

Norstat Group, Germany

Als großes europäisches Feldinstitut bietet Norstat in vielen Ländern alle Methoden der Datenerhebung an. Wir sind überzeugt, dass diese unterschiedlichen Methoden ihre Daseinsberechtigung haben, obwohl, oder gerade weil wir die Stärken und Schwächen der jeweiligen Ansätze gut kennen. In der jüngsten Diskussion um Repräsentativität konnte man jedoch bisweilen den Eindruck gewinnen, dass die unterschiedlichen Erhebungsmethoden gegeneinander ausgespielt werden sollten.

Dieser Vortrag wird zunächst kurz unseren Qualitätsanspruch in der Onlineforschung skizzieren, damit Daten aus unseren Panelbefragungen so repräsentativ wie möglich sind. Dabei gehen wir insbesondere auf die Rolle von Panelrekrutierung, Quotenstichproben und Gewichtung für die Datenqualität ein.

Darauf aufbauend werden empirische Ergebnisse einer Eigenstudie zur schwedischen Parlamentswahl 2018 vorgestellt, bei der die Datenqualität unterschiedlicher Stichproben miteinander verglichen wurde: telefonisch rekrutierte Panellisten versus online rekrutierte Panellisten, jeweils gewichtet und ungewichtet. Die Ergebnisse dieser Studie werfen ein Licht auf die Frage, welchen Einfluss die Rekrutierung von Panelteilnehmern auf die Datenqualität einer Studie hat und ob hier mögliche Verzerrungen durch Gewichtung nachträglich ausgeglichen werden können.

Tress-Gut gewichtet ist repräsentativ genug – Ergebnisse einer Eigenstudie-295.pdf

MRP und Variablenselektion in einer Echtzeit-Anwendung - Eine Fallstudie

Janina Mütze, Tobias Wolfram

Civey GmbH, Germany

Multilevel-Regression und Poststratifizierung (MRP, Park, Gelman and Bafumi, 2004) ist eine immer häufiger genutzte Alternative zu klassischen Design-basierten Ansätzen der Survey-Statistik. Allerdings existieren gewisse Einschränkungen, die die Nutzung dieser Methode erschweren. Hierzu zählen a) die Verfügbarkeit der entsprechenden Poststratifizierungsgewichte, b) das Wachstum der Menge von Poststratifizierungszellen und c) ein potentiell hoher Rechenaufwand. Während mit Blick auf a) große Fortschritte gemacht wurden (Leeman and Wasserfallen 2017), scheinen die Probleme b) und c) die Anwendung von MRP in einer kommerziellen Echtzeit-Anwendung, in der tausende Modelle pro Tag berechnet werden müssen, unmöglich zu machen. Als Lösung dieser Probleme schlagen wir einen zweistufigen Ansatz vor, der hierarchische logistisches Regressionsmodelle und Poststratifizierung mit Variablenselektion durch das LASSO (Tibshirani 1996) kombiniert. Wir präsentieren dieses Modell im Rahmen einer Fallstudie zur Echtzeit-Anwendung von MRP bei Civey, Deutschlands größtem online Access-Panel. Civey nutzt ein Widget-basiertes Empfehlungssystem, welches mehr als 4000 Umfragen über ein Netzwerk von gegenwärtig mehr als 25000 Webseiten ausspielt. Gewichtete Ergebnisse werden den Nutzern in Echtzeit angezeigt und müssen entsprechend häufig neu berechnet werden. Hieraus entsteht die Notwendigkeit zur schnellen Modellschätzung, welche mit einem klassischen MRP-Ansatz nicht möglich wäre. Durch die Anwendung des zweistufigen Verfahrens können wir die Vorteile von MRP mit den Anforderungen einer Echtzeit-Anwendung verbinden. Wir legen die Validität unseres Ansatzes durch den Vergleich mit den Ergebnissen eines vollständigen MRP-Modells dar und diskutieren weitere Herausforderungen und offene Fragen unserer Methode.

4:45 - 5:00Break
5:00 - 6:00A06: Push-to-web and Recruitment
Session Chair: Jessica Daikeler, GESIS - Leibniz-Institute for the Social Sciences, Germany
Room Z28 

Web-push experiment in a mixed-mode probability-based panel survey

David Bretschi, Ines Schaurer

GESIS – Leibniz-Institute for the Social Sciences, Germany

Relevance & Research: In recent years, web-push strategies have been developed in several cross-sectional mixed-mode surveys in order to increase response rates and reduce the costs of data collection. However, pushing respondents into the more cost effective web-option has rarely been examined in the context of panel surveys. This study evaluates how different web-push strategies affect the willingness of mail mode respondents in a mixed-mode panel to switch to the web.

Methods & Data: We conducted a randomized web-push experiment in the October/November wave 2018 of the GESIS Panel, a probability-based mixed-mode panel in Germany (n=5,738). We used an incompletely crossed experimental design with two factors: 1) time of presenting the web-option and 2) prepaid vs. promised incentives by randomly assigning 1,897 mail mode panelists to one of three conditions:

1) the web option was offered concurrently with the paper questionnaire including a promised 10 € incentive for completing the survey on the web,

2) the web option was presented sequentially two weeks before sending the paper questionnaire and respondents were also promised an incentive of 10 €,

3) same sequential approach as group 2, but with a prepaid 10 € incentive instead of a promised incentive.

We examine how conditions differ on the web response rate of mail mode respondents, the proportion of respondents who agreed to switch to the web mode for future waves, and other respondents related variables.

Results: Contrary to our expectation, preliminary results show that prepaid incentives do not affect the web response rate compared to promised incentives. However, there is a tendency that a sequential presentation of the web-option increases the web response rate in contrast to offering the web mode concurrently. Final results of our study will be available in January 2019.

Added Value: This study will provide new evidence on the effect of web-push methods in mixed-mode panel surveys. Our findings may contribute towards better understanding of mode choice and mode switching of participants in probability-based longitudinal studies.

Bretschi-Web-push experiment in a mixed-mode probability-based panel survey-141.pdf

Push-to-web recruitment of a probability-based online panel: Experimental evidence

Ulrich Krieger1, Annelies Blom1,2, Carina Cornesse1, Barbara Felderer1, Marina Fikel1

1SFB 884, University Mannheim; 2Department of Political Science, University of Mannheim

Relevance & Research Question:

Past research has shown that pushing respondents to the web is a successful way to increase response rates, reduce data collection costs, and produce representative outcomes. However, studies in that literature are usually limited to cross-sectional surveys on small and homogeneous target populations. Our study rises beyond this limited scope to a broad and, so far, unique application: We investigate the relative success of pushing respondents to the web compared to alternative survey design strategies across the recruitment stages of a probability-based online panel. In order to do this, we implemented a large-scale experiment into the 2018 recruitment of the German Internet Panel (GIP).

Methods & Data:

In this experiment, we sampled 9,800 individuals from population registers and randomly assigned each individual to an experimental group: online-only, online-first, offline-first, or concurrent-first. Individuals in the online-only group received a postal mail invitation to participate in the web version of the GIP recruitment survey. Nonrespondents in the online-only group were followed up by invitations to the web version of the GIP recruitment survey again. Individuals assigned to the online-first group received the same invitation letter as the online-only group asking them to participate in the web version of the GIP recruitment survey. However, nonrespondents were followed up with a reminder letter containing a paper-and-pencil version of the GIP recruitment survey. Individuals in the offline-first group received the paper-and-pencil questionnaire with the initial invitation letter and were followed up with invitations to the web version of the GIP recruitment survey. Individuals in the concurrent-first group were initially given the choice between participating in the web version of the GIP recruitment survey or the paper-and-pencil version.

Results: Early results show an about 23% percent recruitment rate for the online-first group and lower rates for the other groups. Using paper questionnaires results in higher initial response rates but these respondents but bringing those respondents to the web is challenging.

Added Value: Our research shows the feasibility of postal recruitment to a web panel. We compare different recruitment strategies and their effect on sample composition.

Timing your web survey: Effects of variations in time of contact, respondent’s completion behaviour and data quality outcomes in a course evaluation setting

Ellen Laupper, Lars Balzer

Swiss Federal Institute for Vocational Education and Training SFIVET, Switzerland

Relevance & Research Question: Timing effects in survey research have gained new topicality, as the questionnaire administration via the internet allows contacting all survey participants at the exact same time. Moreover, with the availability of paradata, e.g. starting time and total duration of questionnaire completion, a deeper analysis and understanding of respondents’ completion behaviour is possible. This is especially interesting given the fact that little is known on how timing factors are influencing respondents’ completion behaviour such as non-response, recall effects or response delay in web surveys (e.g. Estelami, 2015; Karapanos, Zimmerman, Forlizzi, & Martens, 2010; Lewis & Hess, 2017). The proposed study wants shed more light on the interplay between survey timing, coverage and recall effects and their effects on data quality.

Methods & Data: A between-subject web survey experiment was implemented in order to examine how different time points of contact (at 8 p.m. on the day of course completion (d0), at 8 p.m. three days after course completion (d3) and at 8 p.m. one week after course completion done (d7)) relate to differences in data quality (e.g. response rate, drop out, item non-response, interview duration, overall course satisfaction, straight-lining and other satisficing behaviour). Around 1000 participants attending one of nine 1-day refresher courses for examiners in vocational education and training (VET) were randomly assigned to one of the three experimental groups.

Results: A mediation model using Mplus was tested with time of contact as a multi categorical predictor, response delay as the mediator and seven data quality indicators as outcomes. It seems that, although sending the course evaluation invitation email directly after the course results in a significantly higher response rate, data quality effects seem to be mainly related to response delay.

Added Value: This study makes an important contribution beyond previous research by bringing together separately researched theoretical considerations and results on survey timing aspects like time of contact, response latency as well as on total response time and thus offers a new view into the possible dynamics of how survey timing affects respondents survey behaviour and as result survey data quality.

Laupper-Timing your web survey-207.pdf
5:00 - 6:00A16: Activities in Online Communities
Session Chair: Hannah Bucher, GESIS, Germany
Room 154 

FemalePathways to Online Pornography – Less Addiction – more Play

Armin Klaps, Lukas Kloss, Jan Aden, Anastasiya Bunina, Zuzana Kovacovsky, Reinhard Ohnutek, Birgit Ursula Stetina

Sigmund Freud Private University, Austria

Relevance & Research Question: Online sexual activities (OSA) are quite common nowadays although still with a male domination and mostly male directed content. But the countermovement is growing. There is pornography especially created for women and there are clear efforts and projects to empower women to enhance their sexuality using online content (eg. OMGYes, a website for women to embrace their sexuality and learn how to have better orgasms).Gender differences in Online Sexual Activities (OSA) seem to decrease over the last years (eg Döring et al, 2017)

Objectives of the presented study was to evaluate a gender balanced sample according to their OSA and clinical aspects such as sexual and internet addiction.

Methods & Data: A sample of 93 online pornography users (51.6% male and 48.4% female) based on postings in pornography related online-groups was surveyed in a cross-sectional design using an online questionnaire including questions about sexual preferences in real life and online as well as several clinical scales such as the Internet Sex Screening Test (ISST, Delmonico, 1997) the abbreviated version of the Sex Addiction Screening Test (SAST-A, Carnes, 1989 accrevated by Delmonico & Miller, 2003) and the Internet Addiction Scale (ISS-20R, Hahn, Jerusalem & Meixner-Dahle, 2014).

Results: Males show on average significantly more problematic online behaviour than females (ISST: T(69.756)=3.823,p<.001) and use online pornography significantly more hours per week (T(41.685)=3.729,p=.001). Women report that they have changed their preferences since consuming online sexual content significantly more than men (T(89.993)=-2.153,p=.034). In addition to that analysis of the proposed ISST factors (Delmonico & Miller, 2003), which only use several items to explain the online sexual behaviour in depth shows that female participants showed significantly more interest in online sexual behaviours (T(71.149)=-3.456,p=.001) show more online sexual compulsivity (T(56.761)=-3.631,p=.001) and more isolated online sexual behaviour (T(85)=-2.965,p=.004).

Added Value: A shift in preferences of women is visible in the current sample. Future studies focusing on womens OSA in a more specific way need to reveal if there is a real trend

Klaps-FemalePathways to Online Pornography – Less Addiction – more Play-200.pdf

Branching Out the Babytree: The Effects of Dual Peer Group Membership on Social Support During Pregnancy in Online Communities

Lingqing Jiang2, Zhen Zhu1

1University of Greenwich, United Kingdom; 2University of Essex, United Kingdom

Relevance & Research Question (Keywords: online community; content analysis):

Social support from peers plays a positive and important role in maternal well-being, previous studies show that lack of social support presents a strong risk factor for maternal depression during pregnancy and the postpartum period which can also impair the mental health of the next generation. Today, the emergence of online social communities provides possibility to seek social support to maternity from a much broader context. Our paper, through text mining of an online community, investigates whether dual peer group membership has positive effects on members social support behavior during pregnancy.

Methods & Data (Keywords: quasi-experimental design; instrumental variable):

We collect a sample data set from the largest online maternity and parenting community in China, (Babytree), which contains over 30,000 pregnant women and over 600,000 online posts in our sample. We use an instrumental variable approach and therefore design a quasi-experiment to explore the effects of dual membership on social support behavior in this online community. We instrument the enrollment of peer groups by the day of users’ estimated due date, which allows a causal interpretation of our results.

Results (Keywords: spillover effects):

We find that the enrollment of an additional peer group of the previous month has positive spillovers on users’ outgoing social support in their default peer group. Channels of knowledge transfer and reciprocity are discussed with suggestive evidence and related policy implications.

Added Value (Keywords: social support; pregnancy; maternity; health):

Our contribution uses a massive set of online data to explore the social support behavior among pregnant women in China. It is an unprecedented scale with rich text information only made possible by recent online technologies. The positive spillover of online interaction with multiple membership will also have significant implications for other virtual or real social platforms.

Jiang-Branching Out the Babytree-186.pdf
5:00 - 6:00B06: Social Media and Online Communities
Session Chair: René Schallner, NIM, Germany
Room 158 

Optimized Strategies for Enhancing the Territorial Coverage in Twitter Data Collection

Stephan Schlosser1, Michela Cameletti2, Daniele Toninelli2

1University of Göttingen, Germany; 2University of Bergamo, Italy

Relevance & Research Question: The use of social media as promising data source has become increasingly important in recent years. Social media data, such as tweets, do not only pave the way for new research possibilities, but also raise completely new methodological and substantial questions in a lot of research field (e.g., social sciences, statistics and so forth). This work aims at finding an efficient and optimized way of data collection. In particular, we compare different data collection strategies in collecting Twitter data (for example in order to enhance the territorial coverage of different geographical areas).

Methods & Data: For this purpose, we collected Twitter data among the whole United Kingdom for a period of 90 days, implementing three different parallel tweet collection strategies, set as follows: 1) the boarders of the 12 UK territorial regions (NUTS) were precisely mapped by means of a large number of medium-sized sub-areas (whereas big cities were covered by many smaller sub-areas); 2) the same borders were mapped as precisely as possible, adapting, at the same time, the size of the sub-areas to the actual population density. 3) A high amount of small and equally sized sub-areas was used in order to map NUTS, without considering the population density. In total, we collected more than 300 million tweets, out of which 1% includes geographical metadata (useful to check the accuracy of data collection’s geo-coordinates).

Results: The analysis of tweets including geographical metadata reveal that these tweets were actually posted in the expected regions. This leads to the conclusion that the same probably happens for tweets without geographical metadata. Moreover, the strategy of population density-adapted sub-areas has proven to cover the posted tweets in the most accurate way.

Added Value: Our findings indicate that, using our second collection strategy, tweets can be correctly assigned to territorial regions, such as cities or country units. Furthermore, we were able to identify an efficient and exhaustive strategy for collecting Twitter data that balances the territorial coverage and the need of dealing with a reasonably sized dataset .

Schlosser-Optimized Strategies for Enhancing the Territorial Coverage-188.pdf

Exploring Instagram Data: What’s in Instagram for Market Research and Social Sciences?

Yannick Rieder1, Simon Kühne2, Daniel Jörgens3

1Janssen-Cilag GmbH, Germany; 2Universität Bielefeld, Germany; 3KTH Royal Institute of Technology, Sweden

Relevance & Research Question: With 1 billion users posting around 100 million photos per day, Instagram has become one of the most relevant social media platforms. Many use Instagram to document their daily life events and share their photos immediately. Thus, Instagram allows for researching social interaction and social phenomena. However, due to Instagram’s restrictive data access policies, almost no research exists that makes use of this valuable data source.

Methods & Data: With access to the Instagram API, we collected over 300.000 posts (photos and accompanying texts) at various time frames in late 2017 and 2018. We focused on selected geographic areas such as Berlin. The data was analyzed using an explorative approach. We apply state-of-the-art text-mining techniques, face and object recognition algorithms and match the geocoded posts with structural data.

Results: Our preliminary results provide insights about the mechanisms of Instagram usage, e.g., most instagrammable places, photo aesthetics and content of posts. Furthermore we identified patterns of social behaviour: What are posting occasions and what influences the event of a post. The analysis will be completed early 2019.

Added Value: We exploit the possibilities of analyzing Instagram data for market research and the social sciences and illustrate examples of applicable research approaches. The application of a broad range of analysis techniques offers insights relating to their quality, costs, benefits and limitations. Finally, general limitations and challenges are discussed with a focus on the upcoming API restrictions recently announced by Instagram.

Rieder-Exploring Instagram Data-137.pdf

The keyboard is the key—Language cues in online dating

Dorothea C. Adler1, Maximilian T. P. Freiherr von Andrian-Werburg1, Frank Schwab1, Sascha Schwarz2, Benjamin P. Lange1

1Julius-Maximilians-Universität Würzburg, Germany; 2Bergische Universität Wuppertal

Relevance & Research Question

Online dating changes how we meet people (e.g., Koch et al., 2005). As gender-typical communication styles are transmitted to cmc (e.g., Guiller & Durndell, 2007) and written cues are used for personality assessments (e.g., Heisler & Crabill, 2006), the choice of words could be particularly important in online dating.

RQ 1: Can the personality of a conversation partner be detected based on a chat?

RQ 2: Are linguistic features linked both to a person’s personality and the receiver’s assessment?

Methods & Data

Two two-step experiments were conducted.

1) Participants completed an online questionnaire assessing several personality traits (study 1: N=189, e.g., Big Five; study 2: N=610, e.g., IQ).

2) Up to 6 participants were invited to the lab (study 1: N=58, study 2: N=116). They chatted anonymously 8 (study 1) / 10 minutes (study 2) with opposite-gender participants and assessed their personality afterwards.

Analyses. The sender’s personality was correlated with the respective assessments. Further, sender’s personality and the personality assessments were correlated with linguistic markers (LIWC; Pennebaker et al., 2007).


Study 1. People assessed sociosexual desire (SD) (r=.33) and IQ (r=.29) correctly. SD correlated with commata (r=-.34), colons (r=-.33), and parentheses (r=-.33). Based on commata (r=-.27) recipients detected SD. A person’s IQ correlated with word count (r=.31), apostrophes (r=.27), and certain topics (anger, r=.28, social aspects, r=-.31, other references, r=-.29, and humans r=-.33). Receivers guessed IQ through words referring to present, r=-.26, achievement, r=.28, money, r=-.30, metaphors, r=.28, religion, r=.31, and death, r=-.28; all ps < .05.

Study 2. People detected openness (r=.20, p=.04), extraversion (r=.17), female (r=.20, p=.04) and male gender role (r=.21, p=.02), and IQ (r=.26, p = .01) correctly (ps = .01).

Added Value

Our studies indicate the signaling character of language and give a first insight on human perception of linguistic markers in online dating. In the presentation, we will discuss the consequences of our findings in terms of their practical relevance for big data approaches as well as with respect to future research.

Adler-The keyboard is the key—Language cues in online dating-149.pdf
5:00 - 6:00C06: Communication on Social Media
Session Chair: Simon Munzert, Hertie School of Governance, Germany
Room 149 

Finnish CEOs in Twitter: Online communication strategies of CEOs with a successful Twitter presence

Laura Liisa Helena Paatelainen1, Pekka Isotalus1, Sanna Ala-Kortesmaa1, Johanna Kujala1, Jari Jussila2

1Tampere University, Finland; 2Häme University of Applied Sciences, Finland

Relevance & Research Question: The research topic is tweeting strategies of Finnish CEOs. The research question is “what kind of strategies do the most followed Finnish CEOs employ to reach a larger audience”. CEOs are increasingly seen as the face of the company they represent and their personal messages can be more effective than the official company messages. CEOs can utilize Twitter for spreading their own message and creating a positive image of themselves and their company. For a tweeting CEO a large follower base is important, as this means a wider audience for their message. Large numbers of followers can also create an image of the CEO as “someone worth listening to”, thus increasing the influence of their message.

Methods & Data: Research data consists of tweets gathered from the public accounts of 133 CEOs of Finnish publicly listed companies and companies on the list of 500 largest companies in Finland. The data was gathered during a seven-month period using Twitter Archiving Google Sheet (TAGS). Research method combines qualitative and quantitative analysis, where tweets are manually categorized by content into categories (topic, function, tone, authenticity and type of interaction). Finally, quantitative analysis is used to determine which categories are the most common among the CEOs with largest follower bases.

Results: Results show the most followed CEOs use a variety of functions to establish a positive image of themselves in Twitter. The most common functions are personal branding, company branding and interaction with stakeholders. The overall tone of these tweets is neutral to positive.

Added Value: So far there has been little research into the Twitter use of CEOs. This research brings added value to the field of leadership communication by providing insight on the communication strategies CEOs employ to gain success in this new platform. So far online communication research has largely focused on political communication in Twitter. This study brings new insight into the field by analyzing how Twitter is used for communication by a different user group. It also benefits business leaders by providing knowledge about tweeting strategies connected to a successful Twitter presence.

Paatelainen-Finnish CEOs in Twitter-166.pptx

Insights from mapping the Twitter network of the German Bundestag

Harald Meier, Arber Ceni

Social Media Research Foundation, USA/Germany/Albania

Relevance & Research Question:

Of the 709 members of the German Bundestag 524 have Twitter accounts, most of which are active. How do members of the Bundestag tweet about one another? What factions can be found? How does party affiliation align with network clustering? Which parties are most central, which are most peripheral? Which politicians are most influential? How do vertex centrality metrics align with party influence?

Methods & Data:

Social network and content analysis is applied to tweets published by members of the current Bundestag over 10-day time frames. The data is drawn from the free Twitter API via NodeXL Pro. Overall network metrics, cluster analysis and vertex centrality metrics are considered to measure, map and compare the changing networks of tweets to reveal the hidden patterns of affiliation and conflict.


We find that internal party cohesion can be measured and varies significantly with Bündnis90/Die Grünen most internally cohesive and CDU/CSU least internally interconnected. The inter party network shows the AfD is least connected and most peripheral within the Bundestag, while content analysis shows that #AfD is one of the most frequently used hashtag across party lines.

The clustering algorithm (Clauset-Newman-Moore) detects different sets of party coalitions when considering different layers of network data and also different time frames: Black-Red (CDU/CSU and SPD), Black-Yellow (CDU/CSU and FDP), Red-Red (SPD and Die Linke), Green-Red (Bündnis90/Die Grünen and Die Linke).

Added Value:

The network approach highlights the importance of using patterns of connection to create clusters of content for analysis in contrast to a single “bag of words” approach. Social media network analysis of content from elected officials can reveal implicit coalitions and highlight leaders and key topics and linked web resources. This method complements traditional methods by offering a comprehensive overview of the structure of connections.

Meier-Insights from mapping the Twitter network of the German Bundestag-225.pdf

Spreading disinformation on Facebook: Do message source or recipient characteristics affect the propagation of ‘fake news’?

Tom Buchanan1, Vladlena Benson2

1University of Westminster, United Kingdom; 2University of West London, United Kingdom

Relevance & Research Question: There is considerable concern about the propagation of disinformation through social media, particularly for political purposes. 'Organic reach' has been found to be important in the propagation of disinformation on social networks. This is the phenomenon whereby users extend the audience for a piece of information. Interacting with it, or sharing it, greatly increases the number of people the information reaches. If some types of people (e.g. those with specific personality profiles) are more likely to propagate misinformation, there is potential for those seeking to disseminate disinformation to extend its reach by selective targeting

Methods & Data: In an online study, 357 Facebook users completed personality and Risk Propensity scales, and rated their likelihood of interacting in various ways with a message posted by either a trustworthy or untrustworthy source. A measure of overall organic reach was derived from these ratings.

Results: Message source impacted on overall organic reach, with messages from trusted sources being rated as more likely to be propagated. Risk propensity did not influence reach. However, low scores on trait Agreeableness predicted greater likelihood of interacting with a message.

Added Value: Findings provide preliminary evidence that both message source and recipient characteristics can potentially influence the spread of disinformation. The scope for those seeking to disseminate disinformation to leverage this effect is a cause for concern.

Buchanan-Spreading disinformation on Facebook-155.pptx
5:00 - 6:00D06: Data Visualization – From Relevant Insights to Meaningful Stories
Session Chair: Florian Tress, Norstat Group, Germany
Session Chair: Oliver Tabino, Q Agentur für Forschung GmbH, Germany
Room 248 

Shiny for interactive data visualization: a case study

Paul Simmering

Q | Agentur für Forschung GmbH, Germany

The Global Patent Explorer is a web application that visualizes information on 8 million patents filed between 1980 and 2016. It was built in Shiny, a package for the programming language R. Researchers, policy makers, and investors can use it to analyze innovation across the world. They can investigate the network structure of patents, and filter by class, time, citation count, and geography on country and city level. It also features novel patent quality indicators from natural language processing of abstracts.

The web application uses reactive plots and user interface elements that update according to user actions. From a visualization standpoint, the main feature is the use of a consistent filtering user interface to control diverse interactive plots, including a map and a network graph.

By example of the Global Patent Explorer, the talk discusses web development in Shiny. The package lets data scientists use the same tool that they use to produce statistical models to build web applications. The talk covers the workflow and data pipeline, the pros and cons of using Shiny, highlights useful R packages, and outlines the economics of development and server costs. It aims to provide practical insights for researchers, firms, and government organizations interested in tools for interactive data analysis on the web.

Donald Says – Visualizing the impact of Donald Trump‘s statements and actions on the news

Marcel Gemander

Bielefeld University, Germany

Relevance & Research Question:
In times of fake news and post-factual politics, the free press has to resist increasingly fierce attacks by the impersonating President of the United States, Donald Trump. How strong is the press reaction to Donald Trump's statements and actions? How does the international online media presence of Donald Trump actually look like and can trends be identified? What are the reactions of the online news landscape on Trumps highly divisive political statements and actions and how can the impact be visualized?

Methods & Data:
The recently developed web based application »Donald Says« automatically collects 146 RSS-Feeds of chosen u.s. american and international online news sites since August 2017 in a 10-minute time interval. The number of Trump articles, current headlines and the distribution of nouns (buzzwords) used provide information about current news developments. Up to the present day, the collected database consists of approximately 30.000 entries.

Firstly, the application visualizes the increase and decrease of published RSS articles utilizing an indexed based time series analyses. The impact of Donald Trumps statements and actions are reflected by the variation of the times series. In order to be able to interpret variations better, the headlines of the 20 most recent Trump articles are also displayed. In addition, the application represents the development and distribution of the nouns used. In this way, thematic focuses and emerging trends can be identified. Moreover, fluctuations of the index and buzzwords are expressed absolutely and as a percentage. By observing the index or numbers the visitor can now easily identify peaks, such as the “midterms peak” on Oktober 7th and the news impact of Jeff Sessions retreat.

Added Value:
The Trump Media Index effectively displays the international scope and media efficiency of Trumps political actions and statements. Additionally, the application provides a fast access to the latest topical background by showing relevant buzzwords and their trends. The public, journalists and companies benefit from a comparison between worldwide news portals in order to better classify political events directly linked to the U.S. president and to simpler indicate possible tendentiousness of regional headlines. Finally, it is easily conceivable to extend the tool to analyse other controversial topics apart from Trump.

Visualisation of Data – Then and Now

Nina Corradini, Paolo Guadagni

The Visual Agency, Italy

Since our earliest beginnings all aspects of our lives have undergone dramatic changes. This is especially true for the way we process and use the information available to us. In order to stay competitive and survive, our ancestors had to respond as quickly as possible to simple information input such as “tiger – danger - flee” or “rabbit – food – hunt”. Nowadays however, we live in a highly complex world and we need to process, analyze and dissect masses of information to understand the world we are living in. To do so, we have to use tools that simplify information and enable our brains to digest and work with the information available to us. This talk will discuss our journey of using the tool of data-visualisation to understand and explain an increasingly complex world. Furthermore it will investigate how data-driven storytelling allows us to communicate complicated matters to an ever growing audience and to find state-of-the-art solutions for the 21st century. Looking closely at some of the very first examples of data-visualisation comparing them to recent examples of highly complex visualisations, this talk’s aim is to showcase the development, importance and application of data-visualisation in a historical and contemporary context.

6:00 - 7:00D16: 4-to-the-floor: Text Analytics
Session Chair: Holger Lütters, HTW Berlin, Germany
Session Chair: Cathleen M. Stuetzer, TU Dresden & DGOF, Germany
Room 248 
8:00GOR 19 Party

Location: Zum Scheuen Reh, Hans-Böckler-Platz 2, 50672 Cologne

The GOR Best Practice Award 2019 will be awarded at the party!

You need a ticket for the party. Drinks and streetfood included. Party tickets are included in conference tickets for all days and Thursday day tickets! No tickets at the door.
Date: Friday, 08/Mar/2019
8:30 - 9:00Begin Check-In
9:00Track A: Internet Surveys, Mobile Web, and Online Research Methodology

Sponsored by aproxima
Room Z28 
9:00Track B: Big Data and Data Science

In cooperation with the International Program in Survey and Data Science (IPSDS)
Room 158 
9:00Track C: Politics and Communication
Room 149 
9:00Track D: Applied Online Research (Angewandte Online-Forschung)

In cooperation with
Room 248 
9:00 - 10:00A07: Mobile Surveys
Session Chair: Joss Roßmann, GESIS Leibniz Institute for the Social Sciences, Germany
Room Z28 

How Do Different Device Specifications Affect Data Collection Using Mobile Devices?

Brendan Read

University of Essex, United Kingdom

Relevance & Research Question:

Previous studies have found differences between surveys completed on mobile devices and desktops. A variety of mobiles devices are being used to respond to surveys. Little is known about how the specifications of mobile devices affect both response behaviour and data quality. This research aims to answer three research questions: What proportions of the variance in different response behaviour and data quality indicators can be attributed to the device and to the participants? Do specific device characteristics impact upon the response behaviour and data quality indicators? Do the observed effects of device characteristics remain after controlling for respondent characteristics?

Methods & Data:

Data is from the Understanding Society Spending Study One, an app based study asking participants to take pictures of receipts or submit information about purchases. This was embedded within the Understanding Society Innovation Panel, a household panel with a probability-based sample representative of Great Britain. The make, model and operating system of the mobile devices were captured. Additional data on device characteristics was collected using Amazon mTurk and web scraping including: RAM, camera quality, screen size, and processor performance.

Cross-classified multilevel models were used to examine the clustering effect of both respondents and devices. Survey outcomes examined include the duration of app uses, the quality of images produced, and the type of submission made.


Results suggest that in some instances sizeable proportions of the variance in survey outcomes can be attributed to the device used. Additionally, strong associations were found between some device characteristics and survey outcomes. Sometimes these associations were stronger than those between respondent characteristics and survey outcomes. Multivariate analyses produced some results that were consistent with device characteristics having a direct effect on survey outcomes. This was more prominent with certain outcomes, particularly the quality of scanned images of receipts.

Added Value:

This paper first demonstrates how collection of additional device characteristics may be carried out. Then the relative impact of devices and respondents is assessed, highlighting the need to consider the devices being used to complete survey tasks at more granular level than simply dichotomising into mobile versus desktop.

Read-How Do Different Device Specifications Affect Data Collection Using Mobile Devices-195.pdf

Does the layout make a difference? An experiment on effects of online survey layout and device on data quality

Ines Schaurer, David Bretschi, Isabella Minderop, Mirjan Schulz

GESIS Leibniz Institute for the Social Sciences, Germany

Relevance & Research:

Nowadays, the majority of online surveys can be defined as mixed-device studies of PC and smartphones. This fact requires rethinking design conventions of web surveys to consider the usage of both device types. Previous studies have mainly focused on either the effect of the device or the effect of the layout on data quality separately. Furthermore, the majority of the studies suffer from the self-selection of a preferred device by the respondents, thus measurement effects cannot be disentangled from selection effects. To overcome this lack of research, we examine the combined effect of devices and layouts on data quality.

Methods & Data:

In an experimental study we applied a 2x3 factorial design to test main effects and interactions of two factors: 1) the device respondents were invited for participation (desktop vs. mobile device) and 2) the presented online survey layout (optimized for desktop vs. optimized for smartphones vs. build-in adaptive layout).

In October 2018 respondents from an online access panel in Germany were randomly invited to one out of six experimental groups. We applied quota sampling regarding age, sex, and education. Overall 3300 respondents finished the survey, what results in about 550 respondents per treatment group.

The experimental design allows us to examine how the treatment groups differ on several indicators of data quality (e.g., break-off rates, duration, response styles, and characters in open-ended questions) and their evaluation of survey experience.


So far, preliminary analyses show that break-off rates are generally higher among smartphone users (9% vs. 19%) and that they are especially high in the group that received the desktop-optimized version of the questionnaire (26%). Contradicting to previous research, the overall survey evaluation does not differ between the treatment groups. Detailed analyses for the more advanced indicators will be available in early 2019.

Added Value:

This study will provide evidence on the effect of layout choices on data quality, depending on the device used. Furthermore, it offers information about comparability of results in mixed-device studies and practical guidance for designing mixed-device studies.

Dispelling Smartphone Data Collection Myths: Uptake and Data Quality in the UK Office for National Statistics (ONS) Large Random Probability Mixed-Device Online Survey Experiments

Olga Maslovskaya, Gabriele Durrant, Peter WF Smith

University of Southampton, United Kingdom

Relevance & Research Question: Social surveys are increasingly conducted via online data collection. They also started embracing smartphones for data collection. In the UK, there is a significant move towards online data collection, including the ambition to move established household surveys such as Labour Force Survey (LFS) as well as the next UK 2021 Census online. Since most online surveys allow participants to respond not only via PC/laptops and tablets but also via smartphones, it is important to understand associated data quality issues. Concerns still exist regarding smartphones producing lower data quality. Also nothing is known yet about longitudinal mixed-device social surveys in the UK. This research is timely and fills the knowledge gaps in these areas.

Methods & Data: We use Office for National Statistics (ONS) data for LFS online experiments (Test 1, Test 1b and Test 2) which were conducted in 2017. The main aim of these experiments was to move LFS online.

Descriptive analysis and then linear, logistic or multinomial logistic regressions are used depending on the outcome variables to study data quality indicators associated with different devices in the survey. The following data quality indicators are assessed in cross-sectional and longitudinal contexts: break-off rates, response latencies, timeout rates, restart rates and differential reporting.

Results: This paper compares data quality between smartphones and other devices in the UK ONS large-scale social survey experiments. It also assesses data quality between devices in the longitudinal context. The good news is that we can be less concerned about allowing smartphone data completion in the contexts where mobile-first design is used for questionnaires as data quality are not very different by devices.

Added Value: The findings from this analysis will be instrumental to better understanding of data quality issues associated with mixed-device surveys in the UK cross-sectional and longitudinal contexts in general and specifically for informing online versions of social surveys and the next UK Census 2021. The results can help improving designs of the surveys and response rates as well as reducing survey costs and efforts.

9:00 - 10:00B07: Opportunities and Challenges of Digitalization
Session Chair: Cathleen M. Stuetzer, TU Dresden & DGOF, Germany
Room 158 

The Variable Harmonization Hub: A case study in Big Data and digital documentation

Kristi Winters1, Inga Brentel2, Martin Friedrichs1

1GESIS - Leibniz Institute for the Social Sciences, Germany; 2Heinrich-Heine-Universität Düsseldorf

Relevance & Research Question: To adhere to the scientific method, researchers combining many datasets must document the harmonization of variables in clear, transparent and replicable ways. At GOR 2018, we presented on: the NRW-Innovativ projects, filling a lacuna in communications studies by creating a harmonized dataset (since 1954) of media use in Germany using the Media-Analysis-Data; the complex process of large-scale data variable harmonization; and how the GESIS software CharmStats Pro assisted in documenting the study, question and variable metadata required for full documentation and in generating recode commands and full reporting documents. This year we will showcase a pre-release of the EU-funded Variable Harmonization Hub (Hub) website, an online repository for storing, searching and downloading digital data harmonization, using the Media-Analysis-Data as a use case.

Methods & Data: We will review the theoretical and practical influence the digital harmonization software, CharmStats Pro, had on the design of the Hub, how the search and download features work, the submission and acceptance for publication project (with a doi). The presentation will include a live demonstration of the Hub using the Media-Analysis-Data as a use case.

Results: The purpose of the Hub is to be an online, digital repository for social science harmonization documentation for long-term preservation, and to make the sometimes complex process of data harmonization both transparent and replicable.

Added Value: The methodological approach of this use case can be counted as big data digital documentation and variable harmonizing for research.

Winters-The Variable Harmonization Hub-208.pptx

Data Literacy in the Age of Insight Democratization

Angelika Satzl, Dominik Racké

Norstat Deutschland GmbH, Germany

Relevance & Research Question:

The age of data democratization is characterized by more and more people having access to data, charts and dashboards. However, it is not always clear if these people have the required skills and abilities to draw the right conclusions from reading them. Consequently, companies run the risk of their employees drawing wrong conclusions and making bad business decisions. Our study assesses the extent of data illiteracy and tries to identify best practices for sharing data with broad and potentially unqualified audiences.

Methods & Data:

We conducted an online survey with 800 German white-collar professionals. Our study covered four different areas: In the first part, we collected baseline figures for the verbal expression of statistical probabilities. We then asked the respondents to evaluate different method descriptions and guess the data quality of the corresponding studies. The third part consisted in a monadic test design to evaluate the readability of different chart layouts. In the last part, respondents could give feedback on the comprehensibility of different dashboard design patterns.


We have not analysed all the data yet, but first insights indicate that many people feel uncomfortable with interpreting charts and have little knowledge about methodology and survey quality. It is also against this backdrop, that many of them draw factually incorrect conclusions or feel uncertain about their interpretations. However, we could also see some design patterns, that improved the accuracy of and confidence in drawing own conclusions for the common user. As stated, a thorough analysis of the mostly open feedback has yet to be made and we expect to identify more best practice principles.

Added Value:

The democratization of insights is a trend that probably will not be stopped. However, we can shape this trend and help to improve the readability of charts. In a broader sense, this may also help to make research become more centric towards the various users of data and insights.

Satzl-Data Literacy in the Age of Insight Democratization-168.pptx

Using Publicly Available Data to Examine Potential Cultural Influence on Concussion Risk in American Football Players

Heidi A. Wayment, Ann H. Huffman, Brian A. Eiler, Patrick C. Doyle

Northern Arizona University, United States of America

Relevance & Research Question: Although concussion risk is widely understood to be a serious health threat, the rate at which American football players report potential concussion symptoms remains low. Low reporting is attributed to a "football culture" that reinforces attitudes and behavior associated with non-reporting. We hypothesized that football programs more invested in "football culture” would have a lower number of concussion reports during the 2017 season. Given the difficulty of assessing cultural influences with traditional self-report measures, we examined whether "program investment and success" (RISC) could be measured using several sources of publicly available data. We hypothesized that RISC would be inversely related to concussion reports.

Methods & Data: Data were collected for all 130 NCAA Division I football programs between September 2017 and January 2018. We identified eight indicators that we theorized would reflect historical success (historic win-loss record, alumni going on to NFL), fan engagement (Twitter, Instagram, and Facebook followers), and financial investment (stadium size, coach salary, ticket price). We also collected 2017 success (win-loss record) and injury data (

Results: Some football program data were publicly unavailable and MI methods were used to replace missing data. RISC indicators were standardized and found to significantly load onto a single hypothesized factor (range: .68 to .88). Items were combined to create factor called RISC, which was reliable (a = .92) and normally distributed. Although there was a small effect, RISC was not significantly associated with concussion injury (r = .15, p < .09). RISC was positively related head and neck injury (r = .30, p < .001) and knee injury (.28, p < .001).

Added Value: Results suggest that RISC may be associated with player increased risk for head and neck injury. The relatively weaker association of RISC to concussion injury is of interest and the subject of follow-up research. We demonstrate the advantages and disadvantages of using publicly available data to provide insight into concussion risk factors attributable to larger, cultural and organizational influence that are typically difficult to measure directly.

Wayment-Using Publicly Available Data to Examine Potential Cultural
9:00 - 10:00C07: Citizens on Social Media
Session Chair: Julian Bernauer, University of Mannheim, Germany
Room 149 

Political Fandoms as Networks on Twitter: Language-Blind Analysis of Polarized Communication

Oul Han, Sarah de Nigris, Matteo Morini

Institute for Web Science and Technologies, Universität Koblenz-Landau

Relevance & Research Question:

Personality-centered politics is a phenomenon in young and old democracies wherever populism and inequality are on the rise. At the same time, digital environments amplify the expression of affective and emotional identification for and against political persons, events, and positions. As result, political fandom—which incorporates anti-fandom—arises online. Using Twitter data, this paper examines whether communication clusters are shaped by ideology or affect, which combines fandom with anti-fandom.

Methods & Data:

The dataset holds tweets in Korean, Japanese, and German. The first step identifies tweet semantics of fandom behavior in non-political subjects and extracts the underlying networks. The second step tests these networks on tweets that regard political subjects. Validation is conducted by comparisons to alternative indices for political and social polarization. Furthermore, the network of Twitter users is examined as well by means of pure network analysis tools, leading to uncover structural changes which, regardless of the language involved, spur a different organization in correspondence with major political events. Human intervention is kept to a minimum, allowing very large datasets, and/or datasets featuring unsupported languages, to be efficiently and quickly processed.


Results show distinct behaviors of fandom: It defends the political idol and simultaneously attacks its projected adversary, regardless of issue. While online fandom cannot represent the offline population, fandom utilizes offline topics for formulating absolute support as well as contention. Online political fandom results from the increase of affective polarization via online media and reveals partisan affect that hates compromise and exchange.

Added Value:

Implications shed light on the fundamental limits of democratic deliberation online, propose a new framework for a global phenomenon that decouples from ideological explanations for political cleavages, and demonstrate a method that can be scaled up across languages and country-specific contexts.

Straight Into the Echo Chamber? How Others' Political Stance Influences Tie Formation in Social Media

Manuel Cargnino1, German Neubaum1, Stephan Winter2

1Universität Duisburg-Essen, Germany; 2Universität Koblenz-Landau, Germany

Relevance & Research Question: Political social media use has become topic to a growing amount of scholarship, inter alia, addressing the role of user behavior in the formation of politically homogeneous environments, so-called echo chambers. Building on the concept of selective exposure, we introduce the notion of selective political friending, i.e., the preference for politically like-minded users in tie formation on social networking sites (SNS). Opinion congruence is contrasted to other criteria of friending, i.e. other users' popularity/supportiveness and career utility and it is analyzed to what extent dispositional variables (i.e., opinion strength, need for cognitive closure, desire for shared reality) influence selective political friending.

Methods & Data: Within a pre-registered laboratory experiment using cover story methodology (N = 199; 40,7 % female; Age: M = 29.48, SD = 11.83 min = 18; max = 75 years), participants were introduced to an ostensible social networking site on which they were asked to select profiles of other users to which they would like to connect (send friendship requests). These profiles contained information on a user's political opinion, popularity and career and were manipulated as within-subject factors. Number of friendship requests sent served as dependent measure.

Results: We found that users preferably build ties with others who hold the same opinion toward controversial political issues. This preference exceeds users' preference for ties that might provide social support or career-related benefits and is particularly pronounced when pre-existing opinions are strong and for morally loaded issues (i.e., refugee family reunification).

Added Value: The present study unravels motivational patterns in the process of tie building on SNS and points to the necessity of taking the motivational complexity into account when studying phenomena linked to echo chambers. Extending selective exposure research, this work reveals that opinion-based selectivity can also occur when it comes to build digital relationships which, in the long run, lead to selecting content that is displayed in one's newsfeed. If others' political opinions are salient or visible, this selective exposure could lead to the formation of echo chambers.

Cargnino-Straight Into the Echo Chamber How Others Political Stance Influences Tie Formation-147.ppt
9:00 - 10:00D07: Data Systems instead of Ad Hoc Research?
Session Chair: Horst Müller-Peters,, Germany
Room 248 

Go beyond ad hoc research: Get inside the mind of consumers

Mathias Friedrichs

GfK, Germany

Thoroughly understanding consumers purchasing behaviors and decision-making lays the foundation for effective marketing, sales, product and category management - in particular for slow moving consumer goods, where purchase cycles are long and consumers investments high. Ad hoc research is a tried and tested approach to inform businesses in this respect. However, it is often not able to meet the requirements of todays’ fast-paced and increasingly digital business environment which is calling for on-demand access to actionable insights. GfK tackles this challenge with the Consumer Insights Engine, a new generation of platform-based analytics solutions: Connecting syndicated ad hoc research with sales data, behavioral data and artificial intelligence in a single integrated platform. This talk will show how to combine data from mobile-first surveys, retail panels, online product reviews and online metering in order to understand consumers purchasing behaviors. It will additionally demonstrate how this data system helps brands and retailers to increase return on investment. Be it by understanding who their customers are, stimulating demand with their messaging, optimizing visibility with their marketing channel mix or providing the optimal customer product experience.

Friedrichs-Go beyond ad hoc research-253.pptx

Plan & Track: Using connected single source data for Campaign planning and performance tracking

Felix Leiendecker

YouGov Deutschland GmbH, Germany

Brands and agencies are currently using different data sets to inform insight, planning and activation, and to evaluate campaign success. Using multiple data sources makes it challenging to compare and combine insights and effectively measure campaign outcomes. The need to prove marketing ROI/ effectiveness and tie it to business outcomes is rising, for both marketers and agencies. So how can brands and agencies plan campaigns, execute (digital) media buying and track campaign performance using a single source of data?

YouGov’s core offering of opinion & passive data is derived from our highly participative panel of 6 million people worldwide who provide us with live, continuous streams of data. With YouGov Profiles, our audience segmentation and media planning tool, users can identify target audiences and describe them in-depth to create personas or dynamic segmentations. Profiles collects and connects data on brand usage, brand perception and brand satisfaction, media consumption, mobile behaviour and social media engagement – all profiled across attitudes, opinions, demographics and elements of lifestyle. Once identified, these target audiences can be bought programmatically in the digital space via our partners (DMP platforms). Through pixel-based retargeting and cookie matching, we can identify which of our panellists has had contact with the ad and follow up via online surveys. Finally, we connect and analyse all of this data in our data analytics tool, Crunch. Since all of the data comes from our panel, it allows stakeholders to speak in a common language around a single source of connected data.

Analyzing Budget Data with Market Research Tools: How Efficient Systems Can Provide More Insights

Holger Geißler1,2, Benedikt Droste3

1DCORE, Germany; 2Datalion GmbH, Germany; 3MSR Consulting Group GmbH, Germany

Many internationally active companies are decentralized and have established complex reporting processes. Typically, data from different locations is forwarded to a central unit for consolidation purposes. This has an impact on the quality of the data and implies a high level of effort for the data centers in terms of consolidation. Furthermore, it creates isolated data sources which cannot be analyzed coherently. This presents a major difficulty for the marketing department: The effectiveness of marketing spendings cannot be analyzed in a reasonable way, since performance data and marketing spendings are not merged in many cases.

DataLion and MSR provide an system to replace and simplify these processes. First, relevant data sources are merged into one database to create a coherent database. Furthermore, manual processes are largely automated to increase data quality and to release resources for content-related work. Afterwards, DataLion offers the opportunity to analyze this data with all instruments of market research by using their their dashboard software. The analysis of centralized KPIs, using methods of market research, creates more transparency and completely new insights into the effectiveness of marketing spendings. The continuous update of data helps to track and evaluate the effectiveness of individual campaigns.

Geißler-Analyzing Budget Data with Market Research Tools-271.pptx
10:00 - 10:30Break
10:30 - 11:15Keynote
Room 69 

Cost-Quality Trade-offs in Web Surveys: Finding the Right Balance

Joseph Sakshaug1,2

1Institute for Employment Research (IAB), Germany; 2University of Mannheim, Germany

Web surveys continue to be a pervasive mode of data collection in the social sciences. Not only are web surveys considered to be cost-effective, they also offer many attractive measurement capabilities that allow for a rich and detailed measures to be collected. However, there is an ongoing debate about whether and when web surveys produce reliable estimates of a given target population, and whether more expensive web survey methods yield more reliable results compared to cheaper ones.

In this talk, I will discuss this debate and highlight empirical examples of cost-quality trade-offs that web surveys face. In addition, I will point to opportunities where mixing cheaper and more expensive web survey methods might lead to improved data quality at a reduced cost.

Joseph Sakshaug is acting Head of the Statistical Methods Research Department and Head of the Data Collection and Data Integration Unit at the Institute for Employment Research (IAB), and Honorary Full Professor in the School of Social Sciences at the University of Mannheim. Previously, he was Associate Professor (Senior Lecturer) in Social Statistics at the University of Manchester and Assistant Professor (Junior Professor) of Statistics and Social Science Methodology at the University of Mannheim. He received his MS and PhD in Survey Methodology from the University of Michigan and BA in Mathematics from the University of Washington. From 2011-2013 he was an Alexander von Humboldt Postdoctoral Research Fellow at the IAB and Ludwig Maximilians University of Munich Department of Statistics. His research focuses on survey design and analysis, combining multiple data sources, and empirical research methods.

11:15 - 11:40Award Ceremony

The following awards will be presented:

- GOR Thesis Award 2019
- GOR Poster Award 2019
- DGOF Best Paper Award 2019
Room 69 
11:40 - 12:00Break
12:00 - 1:00A09: Scales and Don't Know Answers
Session Chair: Katharina Meitinger, Utrecht University, Netherlands, The
This session ends at 1:20.
Room Z28 

When Don’t Know is not an Option: The Motivations behind Choosing the Midpoint in Five-Point Likert Type Scales

Johan Martinsson, Elina Lindgren, Sebastian Lundmark

University of Gothenburg, Sweden

Relevance & Research Question: Likert items, a common attitude measure in surveys, typically has five response categories labelled ‘strongly agree,’ ‘agree,’ ‘disagree,’ and ‘strongly disagree,’ with a midpoint labeled ‘neither agree nor disagree’ to assess an ordered (intermediary) attitude, positioned in equal distance from the points of ‘disagree’ and ‘agree.’ But how do individuals who respond to Likert type questions actually interpret the midpoint value? If respondents select the midpoint for other reasons than expressing a middle position, it violates the assumption of an ordered response scale, and raise questions about the accuracy of the estimates. We investigate how respondents motivate their chose of the midpoint, and how this may vary when ‘don’t know’ is included as response option.

Methods & Data: An online survey experiment was fielded in 2018, with 6,393 members of the Swedish Citizen Panel. All participants were exposed to three 5-point attitude items, split sample with or without ‘don’t know’ as additional option. To assess the reasons for choosing the midpoint option, we asked respondents who selected the midpoint (as well as ‘don’t know’ when included) to motivate why in open-ended questions.

Results: Besides expressing a middle position, we found four general motivations for choosing the midpoint of the 5-point Likert scale; ambivalence, lack of knowledge, no opinion, and indifference. Most frequent were ambivalence and lack of knowledge. The inclusion of ‘don’t know’ as response option yield a lower number of ‘lack of knowledge’ motivations, and an increased number of individuals indicating ambivalence as the reason. However, even when ‘don’t know’ was included, there was a notable share of respondents who still referred to lack of knowledge as reason for choosing the midpoint.

Added Value: The findings comply with previous research, indicating that respondents of Likert type questions choose the midpoint for several reasons besides expressing a middle position. While including ‘don’t know’ as response option has been suggested as a possible solution, however, we find that this alone would not eradicate the problem. Instead, more diverse and item specific measures are likely needed to reduce ambiguities with how to interpret the midpoint in Likert items.

Effects of using numeric instead of semantic labels in rating scales

Tobias Gummer, Tanja Kunz

GESIS - Leibniz Institute for the Social Sciences, Germany

Relevance & Research Question:

Web surveys are increasingly being completed by using smartphones. This trend facilitates the need to optimize question design to fit smaller screens and thereby to provide respondents a better survey experience, lessen burden, and increase response quality. In this regard, it has been suggested to replace semantic labels of rating scales (e.g., “strongly like”) with numeric labels (e.g., “+5”). However, research on the applicability of these scales is sparse, especially with respect to interactions with other scale characteristics. To address this research gap, we investigated the effects of using numeric labels on response behavior in comparison to semantic labels. Moreover, we tested how these effects varied across different scale orientations (positive-negative vs. negative-positive) and scale formats (agree-disagree vs. construct-specific).

Methods & Data:

Our experiment was implemented in a web survey on “Politics and Work” fielded in Germany, November 2018 (N=4,200). The survey was quota-sampled from an access panel. Respondents were randomly allocated in a 2x2x2 between-subjects design in which we varied scale labels (numeric vs. semantic), scale orientation (positive-negative vs. negative-positive), and scale format (agree-disagree vs. construct-specific) of a rating scale comprising 10 items presented item-by-item. The experimental variations were assessed based on several response quality indicators (e.g., agreement, primacy effects, response times, straightlining).


In our preliminary analyses, we found semantic labels to result in more agreement compared to numeric labels. Relying on numeric labels further resulted in larger primacy effects, with primacy effects primarily occurring with construct-specific scales and not with agree-disagree scales. Differences in response times also were moderated by the other scale characteristics. For instance, items with numeric scale labels took longer to answer when scales were aligned from negative to positive but not in the reverse scale orientation.

Added Value:

Our study adds to the sparse knowledge about the usability of numeric scale labels. Moreover, it enhances previous studies by investigating the interaction between different scale characteristics and identifying scenarios in which numeric scale labels may be applied and when they should better be avoided.

Do we know what to do with “Don’t Know”?

Luke Taylor, Tim Hanson, Alice McGee

Kantar Public, United Kingdom

Relevance & Research Question:

Much evidence exists on the treatment of Don’t Know (DK) response options in interviewer administered questionnaires, including arguments around whether they should be explicit options. With the move to online self-completion or mixed-mode designs it is unclear how to best deal with DK and other ‘spontaneous’ codes.

The current approach for online self-completion questions at Kantar Public is to ‘hide’ DK codes and only make them available where respondents try to move on without selecting an answer. Usability testing has uncovered issues with this approach, with respondents often unaware how to select a DK response and feeling forced to select an alternative. This poses questions over whether the current approach risks producing inaccurate data.

Methods & Data:

This paper presents results from an experiment conducted on the UK’s Understanding Society Innovation Panel (IP11) that compared different treatment of ‘Don’t know’ (DK) response codes within a self-completion online questionnaire.

Our experiment compared three approaches:

- Treatment 1 -To ‘hide’ DK codes and only make them available if respondents try to move on without selecting an answer

- Treatment 2 - As above but with a specific prompt at each question on how to view additional options

- Treatment 3 - Including DK codes as part of the main response lists (so they are always visible)


Analysis was conducted across 26 questions. Treatment 3 consistently elicited a higher proportion of DK responses than treatment 1. The differences at self-assessed health measures tended to be small (typically in the order of one or two percentage points). The differences were particularly pronounced for attitudinal questions on low-salience issues.

When asked about the benefits and risks of nuclear energy, only 11% of those exposed to treatment 1 answered DK, compared with 23% for treatment 2 and 33% for treatment 3. Our analysis suggests that treatment 3 effectively discourages the reporting of ‘non-attitudes’. Under treatment 1, 57% of those that reported knowing ‘nothing at all’ about nuclear energy provided a valid (non-DK) answer at the benefits and risks question, this fell to 34% for treatment 2, and 16% for treatment 1.

Taylor-Do we know what to do with “Don’t Know”-172.pdf

The Presentation of Don't Know Answer Options in Web Surveys: an Experiment with the NatCen Panel

Bernard Steen, Curtis Jessop, Ruxandra Comanaru, Marta Mezzanzanica

NatCen Social Research, United Kingdom

While it is possible to collect ‘spontaneous’ answers of ‘Don’t Know’ (DK) in interviewer-administered surveys, this is not so easily the case in self-completion questionnaires. An important decision in web questionnaire design is therefore whether and how to include a DK option. Including a DK option may increase the amount of ‘missing’ data and lead to satisficing, but not including a DK option may negatively affect the data as respondents who genuinely do not know are forced to give a false answer or perhaps exit the survey entirely.

Several approaches have been suggested for web surveys. These include (1) offering a DK option up-front with visual separation from substantive answer options, (2) offering a DK option up-front but probing for more information after a DK answer, and (3) only showing a DK option if a respondent tries to skip a question without giving an answer.

This study tests these approaches by collecting experimental data using the NatCen panel, a probability-based sequential mixed-mode panel. Additionally, the study tests the effects of more explicitly explaining the functionality of option (3) to respondents upfront.

The study looks at the effects of these approaches on the number of DK answers alongside measures of data quality. Further insight is gained into the cognitive processes that respondents went through by using follow-up closed and open probes at the end of the survey to understand why they answered the questions in the way they did.

Steen-The Presentation of Dont Know Answer Options in Web Surveys-194.pdf
12:00 - 1:00B09: Using Smartphone Data for Social Science Research
Session Chair: Anne Elevelt, Utrecht University, Netherlands, The
Room 158 

Process Quality and Adherence in a Mobile App Study to Collect Expenditure Data within a Probability Household Longitudinal Study

Carli Lessof1, Annette Jäckle2, Mick Couper3, Thomas F Crossley2

1Southampton University, United Kingdom; 2University of Essex, United Kingdom; 3University of Michigan, United States

Relevance and Research Question

Spending is hard to measure accurately; respondents may not recall all purchases, they may telescope events and may struggle to estimate totals. Paper-based expenditure diaries are designed to improve reporting accuracy but are burdensome and reported purchases decline over time. In this paper we explore whether data quality is improved using a mobile app which participants use to scan receipts, enter purchases manually or record ‘no spend’ days for a month. We address three research questions:

• To what extent do participants adhere to the spending study protocol?

• Which respondent characteristics and behaviors are related to adherence?

• Does adherence change over the month period?

Methods and Data

The analysis is based on a supplementary study designed to measure household expenditure as part of a nationally representative household panel survey. We invited 2,432 members of the UK Household Longitudinal Study Innovation Panel to download an app and report all their spending on goods and services for a month. Fieldwork took place from late 2016 to early 2017. The paper is based on 268 participants, using data from mobile app entries, scanned receipts, app paradata and covariates from prior study waves and an app registration survey.


We present levels of adherence based on four outcome measures: daily app use, number of purchases reported, proportion of purchases reported by scanning a receipt (rather than entering summary information) and elapsed time between making a purchase and scanning a receipt. We present separate mixed effect multilevel models for each outcome, identifying the key predictors associated with adherence. We model time to examine change in adherence over the study month. Finally, we examine the relationship between the four measures of adherence.

Added Value

This paper recognizes the complexity of full participation in mobile app studies which involve multiple tasks over an extended period. We look beyond initial participation and define and operationalize the more multi-faceted concept of adherence. Our findings will advance conceptualization and implementation of mobile app studies, resulting in better data quality and will ultimately help realize the potential that new technologies offer data collectors.

Lessof-Process Quality and Adherence in a Mobile App Study-180.pdf

The Appiness project - How do (un)happy people behave online?

François Erner

respondi, France

Relevance & Research Question

The Internet has been hailed as a revolutionary tool for bringing people together; sharing knowledge and engaging audiences. But there have also been claims that the Internet has become addictive, increased social isolation and helped to spread hate.

With all that in mind, our objective was to find out if connected technologies are, overall, beneficial or detrimental to our well-being and how.

Over the past decade, measuring and tracking population happiness has become an increasingly important priority for country leaders around the world. The World Happiness Report, supported by the UN, measures well-being all around the world. However, these surveys do not include in-depth analysis about the impact of the digital world on general well-being.

Methods & Data

We conducted this research in order to be able to cross analyse the results of these happiness indexes with online behaviour. Our research - in France, Germany and the UK - combined a traditional online survey, which matched the happiness question wording to the official well-being surveys, with passive tracking data (i.e. web and app behaviour tracked across participant’s phones, tablets and PC/laptops). To obtain real behavioural data was vital here, because when it comes to Internet usage, declarative data may be biased or inaccurate.

Results & Added Value

So what did we find? The key finding was that, as a general rule, the more time you spend on the internet, the less happy you are. This finding was consistent in each of the three countries where we conducted the research.

We also found some fascinating insights when we analysed the happiness of people using particular websites/apps. We are able to depict a cartography of the internet in terms of the happiness score of the audience: the happy and the unhappy zones of the internet.

We are as such able to relate the internet usage by a certain segment of the population to topical issues like “internet addiction”, “social media consumption”, “fake news”… We will also demonstrate/illustrate how this research could be insightful for the market research industry itself.

Erner-The Appiness project-143.pptx

Enriching an Ongoing Panel Survey with Mobile Phone Measures: The IAB-SMART App

Georg-Christoph Haas1,2, Frauke Kreuter1,2,4, Sebastian Bähr1, Florian Keusch2, Mark Trappmann1,3

1Institute for Employment Research; 2University of Mannheim; 3University of Bamberg; 4University of Maryland

The panel study “Labour market and social security” (PASS) is a major data source for labor market and poverty research in Germany with annual interviews since 2007. In January 2018, the supplemental IAB-SMART study has been started, in which selected PASS-participants were asked to install a research app on their smartphones. The IAB-SMART app combines short questionnaires that can be triggered by geographic location with passive data collection on a variety of measures (e.g. geographic location, app use). The triggering of questions allows us to enrich annual retrospective information with data collected immediately after a certain event (e.g. a visit to the local job center). Passive data collection allows innovative measures, e.g. for the integration into social networks via phone and text message logs that complement traditional survey measures. Furthermore, the additional smartphone measures create the potential to address new research questions related to the labor market and technology use (digital stress, home office performance). Finally, the study provides new insights in the day structure and coping behavior of unemployed persons and thus replicate aspects of the classic Marienthal case study from the 1930s with modern means. In this presentation, we will provide an overview of the study and share our experiences in conducting an app project. We will focus on data protection issues, implementation of the fieldwork, participation in the study and participation in short surveys.

12:00 - 1:00C09: Political Communication and Text
Session Chair: Simon Munzert, Hertie School of Governance, Germany
Room 149 

Does the Tail Wag the Dog? The Effect of ECB Communication on Deflation Expectation

Falko Fecht1, Malik Hebbat2, Amirhossein Sadoghi3, Michael Scharnagl4

1Frankfurt School of Finance & Management; 2Deutsche Bundesbank; 3Frankfurt School of Finance & Management,Hohenheim University; 4Deutsche Bundesbank

In this paper, we empirically investigate how communications by the European Central Bank (ECB) alter the market’sbeliefs about the market deflation outlook. We determine events when the Executive Board of the ECB express theirviews about deflation in the Eurozone within the observation period from October 2009 to October 2016. In doingso, we apply a textual analysis of the contents of formal ECB communications as well as related news and Tweets.We use options on the inflation index to extract market deflation expectations. In this deflationary environment, werun our quasi-natural experiment to understand the ability of ECB to shape market expectations through their voices.Our main findings show that within a Granger causality relation between traditional media and social media, formalECB outlooks of deflation have diverse effects on market deflation expectations. The long-term and short-term marketdeflation expectations are mostly derived by high influential members of social media and online type of news mediawhich carry the ECB messages, respectively. Taken together our results suggest that ECB communications underminethe anchoring of inflation expectations in the Euro area.

Fecht-Does the Tail Wag the Dog The Effect of ECB Communication-173.pdf

Cross-Lingual Topical Scaling of Political Text using Word Embeddings

Julian Bernauer, Federico Nanni

University of Mannheim, Germany

Relevance and Research Question:

While the measurement of political positions from text has a long tradition in political science, the rapid developments in machine learning and natural language processing offer new opportunities to extract more information. Especially, it is now much easier to move beyond a bag-of-words approach at the large scale, extracting more information from political text. The article proposes an application and evaluation of cross-lingual topical scaling on political manifestos, asking whether it allows a valid measurement of populist rhetoric from political text.

Methods and Data:

Populist rhetoric is estimated relying on several hundred election manifestos (source: from eight European countries across five languages (English, French, Spanish, Italian and German). We adopt word embeddings, which are vector representations of the meaning of words in their context. Using Python, these are utilized to assess the similarity of sentences from election manifestos to a set of core populist keywords, aggregating at the manifesto level using harmonic function label propagation. The results are assessed for face, convergent and construct validity. The approach features cross-lingual word embeddings and additional model tweaks to accommodate differences in populist discourses across contexts.


Analyses on parts of the sample (German and English manifestos) reveal that the approach works reasonably well within single countries. The resulting measure places well-known populist parties at the correct end of the spectrum, correlates with existing data and detects diverse populist statements in manifestos based on a simple query on word embeddings. At the same time, the cross-lingual scaling exercise appears challenging, with country clusters emerging from the analysis.

Added Value:

In addition to moving beyond a bag-of-words approach, the paper uses word embeddings for topical scaling in a way that does not require the qualitative coding of a training set. If the approach proofs valid, it can safe even more human effort in the estimation of levels of populist – or other – rhetoric in political text. The method delivers continuous measures of sparse, here populist rhetoric for all kinds of parties which can be utilized in the comparative analysis of politics.

Bernauer-Cross-Lingual Topical Scaling of Political Text using Word Embeddings-235.pdf
12:00 - 1:00D09: Digitalization in Qualitative Research: Opportunities, Limitations
Session Chair: Edward Appleton, Happy Thinking People GmbH, Germany
Room 248 

Using VR for Focus Groups: Risks and Rewards

Michael Björn

Ericsson ConsumerLab, Sweden

This talk will focus on experiences Ericsson ConsumerLab have had with using virtual reality (VR) as a means to conduct actual focus groups. Rather than bringing participants to a central location and exposing them to various VR tests, this instead means designing a focus group room inside of VR and then inviting participants who already have their own VR headsets to virtually attend focus groups sessions from wherever they happen to be physically located.

Initially, we thought that recreating focus groups in VR would be relatively straightforward, but there was a wide range of challenges, many of which were related to participant behaviour and the fact that there are no established social norms for this type of social setting in VR. Virtual body language also turned out to be very expressive, yet different from physical body language.

Recruitment of respondents was complicated as traditional agencies could not recruit the user base we were looking for. There were also technical platform issues we needed to work with, not so much related to the VR environment used as related to the users’ own equipment.

Finally, we will discuss the pro’s and cons with focus groups in VR and how they might be a very interesting complement to traditional face-to-face sessions going forward. Despite a low initial user base that limits the type of topics suited for VR focus group work, we believe cost savings and efficiency gains will make VR an important qualitative tool going forward.

Out With Words: Are Pictures the New Black?

Anton Kozka, Sarah Jin

Happy Thinking People GmbH, Germany

Quantitative researchers work with lots of numbers, whilst qualitative researchers work invariably with lots of words, written and spoken. But what about pictures? Instagram – the social media platform based around picture sharing, has over 1 billion active users, uploading 100+ million photos every day, which receives 4.2 billion likes daily. As researchers we can’t ignore such an enormous phenomenon, the explosion of the visual - so we set about exploring it via a radical experiment to find out what would happen when we conduct a piece of mobile research replacing words entirely with pictures.

For our first experiment we used pictures for the questions, the participant responses and the analysis – cutting out all words. The responses we received back were full of personal storytelling, but they lacked one important element necessary for consumer understanding: context. This lead us to conduct a modified follow-up experiment, replacing regular pictures with “augmented pictures” – pictures with levels of embellishment, as widely created and shared on Social Media.

The findings were extremely insightful, and gave us some food for thought for the way we do research – a reminder that the research world and the real world are not two planets existing in silos. Consumers shouldn’t change their behaviour to market research requirements – it’s us researchers who need to adapt our methodologies to reflect consumers’ worlds. Whether that’s using pictures, augmented pictures or otherwise – in order to understand consumers fully, we must do what we can to get close to them, adapting our approaches in synch with changing communication habits.

(Wo)man vs. Machine: If, how, and when to automate Qualitative Research

Julia Görnandt

SKIM, Germany

It is no secret that the market research industry is under pressure to deliver sound and strategic insights within shrinking budgets and time frames. Ten years from now efficiency is predicted to be the number one deciding factor when commissioning research. This presents a serious challenge for qualitative researchers whose traditional methods are not compatible with this need for speed. Whilst demand for qualitative research may be growing, unless we find a way to adapt our processes there is the risk of becoming irrelevant. Confronted with this scenario, we have set out to discover if automation and AI in qualitative research – especially in analysis - is even feasible, and if so, what are the benefits and drawbacks for research professionals? What is the impact on the time, cost and quality of insights? Moreover, how do clients evaluate these trade-offs? To answer these questions we collaborated with Danone, devising a head-to-head competition between human analysis, machine analysis and a combination of the two. This resulted in three research reports, evaluated by Danone in relation to their business needs. From this experiment we learnt that the outputs from machine and AI analysis do not offer ‘magic bullet’ insights. Standalone, automation delivers little value and is limited in its sophistication. Yet, if viewed as a facilitator, automated tools can certainly be used to our advantage during the ‘human’ qualitative analysis process. Thus, by using these, a full report was produced in half the time versus a full report created by human analysis alone. Moreover, our client evaluation confirmed that the time and cost benefit incurred did not compromise the quality of insights. In fact, this report was selected as the preferred option prior to revealing the methodology behind each one.

Görnandt-(Wo)man vs Machine-257.pdf
1:00 - 2:15Lunch Break
2:15 - 3:15A10: Learning Effects, Recall, and Panel Conditioning
Session Chair: Bella Struminskaya, Utrecht University & DGOF, Netherlands, The
Room Z28 

Dynamics and moderators of panel conditioning effects. A meta-analysis.

Tanja Burgard1, Michael Bosnjak1, Nadine Kasten2

1ZPID - Leibniz Institute for Psychology Information, Germany; 2University of Trier, Germany

Relevance & Research Question:

Panel Conditioning is a learning effect, that can endanger the representativeness and validity of results from panel studies. It describes the change in attitudes or behaviors themselves or the way they are reported due to the participation in former survey waves. The meta-analysis examines which moderators affect the strength of panel conditioning. Moreover, the development of panel conditioning over time will be investigated.

Methods & Data:

The literature search was conducted using the broad search interface CLICsearch. To be included, articles had to report randomized or quasi-experiments, involving a control group of fresh respondents or actuary information from a registry and at least one group of conditioned respondents. Both groups had to be exposed to identical survey questions to enable between-group comparisons of quantitative survey outcomes. 20 studies met these criteria.

Data was collected on four levels: First, general information on the report; second, information on the sample composition and conduction of the study; third, information on the kind of intervention, such as incentives or conditioning frequency; finally, the outcome measures for the differences between the control group and a corresponding treatment. The effect sizes used for the meta-analysis are standardized mean differences.

Using the metafor package in R, four-level mixed effects models will be used to meet the needs of the hierarchical data structure. To test the time effect, the influence of the year of data collection on the strength of panel conditioning will be tested. Afterwards, further characteristics of the intervention are tested as moderators.


The first calculations indicate, that the type of question is the moderator with the greatest impact on the strength of panel conditioning. Knowledge questions suffer the most from panel conditioning, followed by attitudinal questions. A time effect concerning the year of data collection cannot be detected with the available data.

Added Value:

The meta-analysis will reveal which kind of questions are particularly affected by panel conditioning. Recommendations on the implementation of panel surveys, such as the optimal frequency and time intervals between waves, will be concluded.

Burgard-Dynamics and moderators of panel conditioning effects A meta-analysis-139.pdf

Recalling Survey Answers: A Comparison Across Question Types and Different Levels of Online Panel Experience

Tobias Rettig1, Jan Karem Höhne1,2, Annelies Blom1

1University of Mannheim; 2RECSM-Universitat Pompeu Fabra

Relevance & Research Question:

Measuring attitudes, behaviors, and beliefs over time is an important strategy to draw conclusions about social developments. The use of longitudinal study designs is also important to evaluate measurement quality (i.e., reliability and validity) of data collection methods. However, one serious concern associated with repeated survey measurements is that memory effects can affect the precision of parameter estimations. So far, there is only a small body of research dealing with respondents’ ability to recall previous answers. In this study, we therefore investigate the ability of respondents to recall their answers to previous questions.

Methods and Data:

We conducted an online survey experiment defined by question type (i.e., attitude, behavior, and belief) in the November 2018 wave of the probability-based German Internet Panel. To evaluate respondents’ recall ability, we employed follow-up questions asking whether they recall their answers, what their answers were, and how certain they are about recalling their answers.


The results indicate that respondents recall their answers, irrespective of the question type. Interestingly, respondents are more likely to recall answers to behavior questions than to attitude or belief questions. In addition, respondents who give extreme answers are much more likely to recall their answers.

Added Value:

Our empirical findings indicate that respondents have a high recall ability. Consequently, the precision of parameter estimations is a serious concern in studies with repeated survey measurements.

Looking up the right answer: Errors of optimization when answering political knowledge questions in web surveys

Jan Karem Höhne1,2, Carina Cornesse1, Stephan Schlosser3, Mick P. Couper4, Annelies Blom1

1University of Mannheim, Germany; 2RECSM-Universitat Pompeu Fabra, Spain; 3University of Göttingen, Germany; 4University of Michigan, USA

Relevance & Research Question: Political knowledge is an important determinant affecting outcomes in public opinion research and political science, which can have a profound impact on governmental decision-making processes. However, some respondents look up the right answer (e.g., on the Internet), which inflates political knowledge scores and can be seen as a kind of “optimizing error” (Yan, 2006) committed by engaged respondents with good intentions. As indicated by previous research, this response behavior is detectable in web surveys using indirect methods. In this study, we investigate optimizing errors when answering political knowledge questions in web surveys by using paradata. More precisely, we use JavaScript “OnBlur” functions enabling us to gather whether respondents switch away from the web survey to search for the correct answer on the Internet using the same device.

Methods & Data: We conducted a web survey experiment in a German non-probability access panel (N = 3,332) and used a two-step split-ballot design with four groups defined by device type (i.e., PC and smartphone) and question difficulty (i.e., open and closed response format). Our expectation is that looking up the answer is more likely on PCs and open response formats. Additionally, we measured response times in milliseconds, employed self-report questions, and measured several respondent characteristics.

Results: The preliminary results indicate that respondents indeed switch away from the web survey page to search for the right answer on the Internet. This finding is supported by the JavaScript “OnBlur” functions and by respondents’ self-reports. In line with our expectations this is more common on PCs and open response formats.

Added Value: The findings provide new insights on optimizing errors when answering knowledge questions. Furthermore, they reveal that paradata seem to be a promising way to observe response behavior that may lead to incorrect inferences about respondents’ knowledge measured in web surveys.

2:15 - 3:15B10: Mobility and Activity Data from Smartphones
Session Chair: Emily Gilbert, University College London, United Kingdom
Room 158 

What Really Makes You Move? Identifying Relationships between Physical Activity and Health through Applying Machine Learning Techniques on High Frequency Accelerometer and Survey Data.

Joris Mulder, Natalia Kieruj, Seyit Höcük, Pradeep Kumar

CentERdata - Tilburg University

Relevance & Research Question

Physical activity is an important indicator of health, but an accurate and objective measurement of physical activity is needed to gain insight and understanding of what drives differences in physical activity and how this influences health. Existing studies are generally based on self-report surveys and while the results of these studies are valuable, there are limitations to their use, e.g., varying perception of physical activity, social desirable answers, and incomplete recall of activity. Using wearable accelerometers as a measurement device provide a more complete and objective picture of physical activity and opens up new ways to study the relationship between physical activity and health. In addition, the influence of socioeconomics, -demographics and personality traits can be taken into account when studying these relationships.

Methods & Data

To study these relationships in detail, an experiment using accelerometers was conducted in the Dutch LISS panel. 1.000 panel members were invited to participate in the experiment. They were asked to wear an accelerometer for 8 days, measuring their physical activity level day and night. During this period, respondents regularly filled out surveys indicating what specific activities they conducted during the day. Furthermore, they provided details about their sedentary behavior, perceived health, the social context, and their associated mood.


First analyses of the objective accelerometer data and the subjective self-reported data showed how people tend to over-report physical activity compared to the objective measurements from the accelerometers. Moreover, these variations differ across socioeconomic and demographic groups (Kapteyn et al., 2018).

Added value

In a follow-up study we take the analyses of these high frequency data a step further by applying data science methods. Using machine learning techniques, such as deep (convolutional) neural networks for pattern recognition, clustering analyses, and classification, we are able to identify specific patterns, i.e. walking, running, cycling, sitting, sleeping. When we combine this with longitudinal survey data from the LISS panel we obtain relationships between physical activity and health on a detailed level. In essence, we gain insight in the relationship between the specific identified activities and personality traits, health, and socioeconomic and demographic status.

Mulder-What Really Makes You Move Identifying Relationships between Physical Activity and Health through.pdf

Squats in surveys: the use of accelerometers for fitness tasks in surveys

Anne Elevelt1, Jan Karem Höhne2,3, Annelies Blom2

1Utrecht University; 2University of Mannheim; 3RECSM-Universitat Pompeu Fabra

Relevance & Research Question:

Smartphones are becoming increasingly important and widely-used in survey completion. Smartphones also offer many new possibilities for survey research, such as extending data collection by using sensor data (e.g., acceleration). Sensor data, for instance, can be used as a more objective supplement to health and physical fitness measures in mobile web surveys. In this study, we therefore investigate respondents’ willingness to participate in fitness tasks during mobile web survey completion. In addition, we investigate the appropriateness of acceleration data to draw conclusions about respondents’ health and fitness level.

Methods & Data:

We used “SurveyMotion (SM),” a JavaScript-based tool for smartphones to gather the acceleration of smartphones during survey completion and additionally employed traditional health and physical fitness measures. We asked respondents if they would generally be willing to take part in a fitness task during mobile web survey completion and employed a subsequent fitness task in which we asked respondents to do squats (knee bends) for one minute. Thus, we investigate respondents’ hypothetical as well as actual willingness and the general comparability of acceleration data with established health and physical fitness measures. We conducted an observational study by using a German nonprobability-based web panel with n = 1,600 respondents; the data collection takes place in September 2018.


59.3 % of the respondents expressed hypothetical willing to participate in a fitness task, 56.7% actually participated in the squat task. The acceleration data are currently being prepared for analyses, so results about its usefulness and comparability to self-reports of respondents’ health and fitness level will be available soon and will be presented at the GOR conference.

Added Value:

This study contributes to the development of more objective measures of respondents’ health and fitness in mobile web surveys and could be extended by further physical activity tasks in future research.

Marienthal 2.0: Research into the subtle effects of unemployment using smartphones

Sebastian Bähr

Institute for Employment Research (IAB), Germany

Unemployment has serious consequences for the lives of those affected. The classical Marienthal study from the 1930s already used innovative data collection methods to analyze the subtle effects of unemployment. The participating observation took on a special significance in the study design. Famous is the finding of a reduced walking speed with prolonged unemployment, caused by the loss of the daily structure.

Nowadays, smartphones provide researchers with new data sources to analyze the subtle effects of unemployment. Innovative tools, such as the sensors built into the smartphone, and at a previously unknown frequency provide unique data to update the Marienthal-findings.

In 2018, the Institute for Employment Research (IAB) conducted the IAB-SMART study. Respondents of the panel study “Labour market and social security” (PASS) were invited to install the IAB-SMART app on their Android smartphone. In addition to surveys, sensor data from the devices were collected via the app. Depending on the consent of the respondents, this included location information and data acceleration sensors or step counters. Similar to the Marienthal study, these passive measurements take place without conscious perception of the interviewees.

Smartphone data makes determining the mobility of the respondents in everyday life possible, such as the choice of means of transport, number of steps and also the respective speeds more precisely and without effort for (and with reduced influenceability by) the respondents. The link to PASS and the administrative data of the Federal Employment Agency allows this information to be linked to the labor market behavior of the persons.

2:15 - 3:15C10: Privacy and Trust
Session Chair: Manuel Günter Cargnino, Universität Duisburg-Essen, Germany
Room 149 

The impact of GDPR on political research

Luke Taylor

Kantar Public, United Kingdom

Relevance & Research Question:

Under GDPR Article 9(2) political opinions are classified as special category data (sometimes known as sensitive data) and explicit consent is required from data subjects to process this data. Guidance from the UK Market Research Society is that questions relating to voting can only be asked to individuals that have explicitly consented to responding on this topic.

This paper examines whether offering respondents the chance to opt-out from answering voting questions potentially reduces the accuracy of political research.

Methods & Data:

Kantar Public does regular political opinion polling in the UK. This analysis is based on five online opinion polls conducted between June and November 2018 (n=6,072).

All respondents were asked demographic questions and questions regarding the economy, Brexit, and policy issues. Questions relating specifically to voting behaviour – both in the past and future intentions – were only asked to the sub-set of respondents (92.6%) that consented to answer these.

The ‘ask all’ questions have been analysed to determine whether those that chose not to answer the voting questions are systematically different from those that consented.


The following analysis controlled for demographic differences (age, gender, region, education and working status) between respondents that consented to the voting questions (n=5,623) and those that opted out (n=449).

The response given to the consent question was found to be significantly associated with:

• Self-reported job security (chi-square p<0.01)

• Economic sentiment (rating of the economy now vs 12 months ago - chi-square p=0.05)

• The order in which respondents ranked eleven policy areas, in particular “reducing unemployment” and “increasing healthcare spending”

These attitudes are likely to be correlated with political opinions, and this suggests that the voting intentions produced from these polls may be biased.

Added Value:

This paper seeks to help inform the methods used for future political research. The findings indicate that complete case analysis is likely to be biased even with demographic weighting applied. Further research is required to determine the best way in which to deal with this.

Taylor-The impact of GDPR on political research-201.pptx

Linking survey data with social media data and the importance of informed consent

Johannes Breuer, Sebastian Stier, Pascal Siegers, Tobias Gummer, Arnim Bleier

GESIS - Leibniz Institute for the Social Sciences, Germany

Relevance & Research Question: Researchers in the quantitative social sciences have traditionally relied on survey methods for studying online behavior. In recent years, however, there has been increased interest in the use of so-called digital trace data, in particular data from social media platforms. Both types of data have specific limitations. Hence, combining these two data sources holds great promise for social scientific research on online behavior. Linking survey data and social media data at the individual level, however, requires explicit and informed consent from study participants. Especially in Europe with the recent introduction of the GDPR, researchers have to provide detailed information about what data they collect, how they collect it, and for what purpose.

Methods & Data: We conducted a web survey with participants of a German online access panel in which browsing behavior was tracked. Of 2042 invited panelists, N = 1347 completed our online questionnaire that focused on political attitudes and behavior and media use.

Results: 22.8% of the respondents (n = 307) reported having a Twitter account, and of those, 65.8% (n = 202) consented to the collection of their Twitter data. 196 supplied a Twitter handle, of which 68 were unusable due to typos, invalid strings or accounts that belonged to somebody else (e.g., celebrities). In a logistic regression model, we found that male, younger, and lower income Twitter users as well as those who have recently used Twitter are more likely to consent, whereas education and the incentive condition had no effect. Specifically, we compared a 5 Euro prepaid to a 5 Euro postpaid condition. Interestingly, only a very small minority of the respondents read the extended privacy information that was provided on the project website via a link embedded in the short version of the short informed consent in the web questionnaire.

Added Value: Our study highlights challenges that need to be considered when linking social media data and survey data. Moreover, we provide a critical perspective on obtaining consent and gathering information needed to link both sources for panelists of online access panels—populations that are frequently used to study consent decisions and generalize findings.

Breuer-Linking survey data with social media data and the importance-210.pdf

When Passion Meets Technology: Enthusiasm Influences Credibility and Trustworthiness in Online Health Forums

Lars König, Regina Jucks

University of Münster, Germany

Relevance & Research Question:

Current scientific debates, such as on new technologies and their application, often involve emotional rhetoric styles. Those who read or listen to these kinds of scientific arguments have to decide whom they can trust and which information is credible. How do information seekers make such decisions?

Methods & Data:

Using a 2x2 between-subject online experiment, the current study investigates how the language style (neutral vs. enthusiastic) and the professional affiliation (scientist vs. lobbyist) of a forum post author influence his trustworthiness and the credibility of his information.


Results show that lobbyists were perceived as more manipulative than scientist. Furthermore, if the forum post author used an enthusiastic language style, he was perceived as more manipulative, less knowledgeable and his information was perceived as less credible. Moreover, both experimental factors interacted: When the forum post author was a scientist, enthusiastic language led to lower benevolence and integrity ratings. For lobbyists, this effect did not occur.

Added Value:

Since most of the Internet is not governed by editors, the validity of online information cannot be guaranteed. The current study demonstrates that information seekers use language characteristics to decide whether they should accept scientific arguments they encounter in online forums.

König-When Passion Meets Technology-214.pdf
2:15 - 3:15D10: Data Science: Bringing Data to Life – Three Applicable and Inspiring Approaches
Session Chair: Yannick Rieder, Janssen-Cilag GmbH, Germany
Room 248 

Network Analysis – A Neglected, but Highly Predictive Source for Consumer Insight

Stefan Oglesby

data IQ AG, Switzerland

Network analysis is an established method in quantitative social research. Back in 1961, James C. Coleman published a hand-drawn network of girls in a local school to illustrate the importance of networks and the individuals’ position within the network with regard to social behaviour. An experiment on anti-conflict intervention in 56 New Jersey public middle schools demonstrated, that interventions through “social referents”, i.e. students with a dense social network, are more effective than interventions addressed to all students irrespective of their role within the school’s social network.

This kind of network analysis is time-consuming and requires expensive data collection. Only with the advent of social media – or better social network platforms – it became possible to collect data on social networks and on the actual content of conversations between the “actors” at a large scale.

Indeed, combining social network analysis (SNA) with automated text analysis has the potential to understand how public opinion , marketing messages, brand preference or consumer expectations and attitudes spread throughout a target audience. SNA is particularly helpful to understand the role of influencers or opinion leaders for the adoption of new products or brand preference in specific markets.

Two case studies combining SNA with more traditional metrics of brand preference show that the heterogeneity or homogeneity of an individual’s network with regard to a specific product category indeed explains a large part of preference – both in B2B and B2C markets.

Deep Learning – Decision Making Made Easy?

Daniel Jörgens

KTH Royal Institute of Technology, Sweden

During the recent decade, developments in the field of GPU computing paved the way for the rise of Deep Learning (DL). Since then, this powerful Machine Learning (ML) technique has been proven to outperform humans as well as state-of-the-art algorithms in various fields of application. However, difficulties in acquiring suitable datasets for training or special reliability requirements can render the utilisation of DL methods challenging for researchers in certain application areas.

In this talk, I will introduce the general concept of DL and challenges arising thereof in the specific context of Biomedical Imaging. Starting off with the view of DL as a 'black box', I will contrast the paradigm of DL methods with classical algorithms in the field and point out the arising consequences for interpreting results. In particular, the definition of a so-called ‘ground-truth’, i.e. a reference for training and validation of a DL method, is often a difficult task. In view of that and the limited traceability of DL-based algorithms, the question of how such methods can potentially support doctors in daily clinical practice also comprises an ethical component.

All in all, this presentation should contribute to the discussions around the question: Are we able to fully comprehend a powerful tool like Deep Learning in order to use it for making crucial decisions.

Do German job advertisements differentiate between men and women? How gender-specific language consolidates gender inequality.

Daniel Spitzer

100 Worte Sprachanalyse GmbH, Germany

Although gender equality is demanded in work-place across industries, the reality in many German companies is different. Women are clearly underrepresented in many occupations. Gender stereotypes are widespread and well documented in social psychological literature. In general, women are described as more social and connected than men, while men are more associated with leadership and activity. Numerous studies on the use of language by men and women have revealed significant differences between the both. For instance, women are said to use more social words, more emotional words or more words concerning relationships compared to men. These findings aroused our curiosity and we asked ourselves whether discriminatory gender effects could be found in job advertisements in the German job market.
To get to the bottom of this question, we collected over 32,000 job advertisements from online job boards and divided them into occupations with a high proportion of men or women. Finally, we examined the language of these job advertisements using the 100 Worte text analysis. The results were clear: job advertisements in male-dominated occupations contained more masculine words than those in female-dominated occupations. The same applies - even to a greater extent but in the opposite direction - to female-dominated occupations. We were thus able to replicate the findings of other researchers and, for the first time, prove them for the German region. Based on these findings, we come to the conclusion that there are structural inequalities in the language of job advertisements that perpetuate existing gender differences.

Spitzer-Do German job advertisements differentiate between men and women How gender-specific language.pdf
3:15 - 3:30Break
3:30 - 4:30A11: Methods to Improve Questionnaires
Session Chair: Stephanie Gaaw, Technische Universität Dresden, Germany
Room Z28 

Context Effects in Online Probing of Sensitive Topics – Explorations Using Survey Data and Paradata

Patricia Hadler

GESIS - Leibniz Institute for the Social Sciences, Germany

Relevance & Research Question

Surveys interested in socially undesirable behavior are likely to include questions on the behavior itself and related attitudes. Context effects have frequently been analyzed for sensitive topics in a survey context. However, little is known about their impact on cognitive pretesting. Online probing offers an anonymous and self-administered setting to study these effects. Using questions about delinquency, this study examines the impact of question order on probe response for sensitive behavioral and attitudinal questions. Analyses combine survey and probe answers with client-side paradata.

Methods & Data

A behavioral and attitudinal question were randomized and each followed by an open probe. A client-side paradata script collected item-level response times, answer changes, page revisits, time spent answering the probe and corrections to probe response. Over 320 respondents in Germany and the US participated in the survey via an online access panel.


Probes following behavioral questions show a higher level of non-response. However, whereas these answers do not differ strongly with question order, respondents report a more lenient attitude in the probe following the attitudinal question when the prior question asked about their behavior. Moreover, probes following a behavioral question are likely to include both behavioral and attitudinal content, whereas probes following an attitudinal question generally do not reference past behavior.

The length of probe response varies with question order for the probe following the behavioral question only. Attitudinal probe responses are associated with more text corrections in both probes. Response times for the probe mainly depend on the answer given to the survey question and the content of the probe. They correlate strongly with the response time of the survey question for respondents giving non-substantive answers.

Added Value

The study analyses context effects in online pretesting of sensitive topics by combining survey and probe responses with paradata from both. Results strongly support the notion of testing question order during pretesting, as question order impacts the results of both online probing and survey response.

Taking Respondents Seriously: Feedback in Mixed-Device Studies

Katharina Meitinger1, Henning Silber2, Jessica Daikeler2, Christoph Beuthner2

1Utrecht University; 2GESIS Leibniz Institute for the Social Sciences

Relevance & Research Question: Feedback questions evaluating respondents’ satisfaction with a survey are an important source of information for survey designers to improve surveys and predict future participation behavior. When asked as a closed-ended question, they provide quick to analyze, standardized insights into respondents’ satisfaction with relevant aspects of a survey. When asked as an open-ended question, the analysis is more time-consuming but the qualitative date can disclose valuable in-depth information, e.g., whether respondents faced previously undetected technical problems as well as formatting issues or gain information which items were affected by social desirability issues. Additionally, feedback questions are particularly important in mixed-device studies since mobile respondents might encounter particular challenges (e.g. suboptimal visual design) when responding to a web survey.

However, there are several research gaps regarding the optimal design of feedback questions in mixed-device studies: 1) It is unclear whether mobile respondents provide comparable response quality as PC respondents. 2) It is unclear whether a question order-effect occurs when using a combination of closed and open-ended feedback questions.

Methods & Data: An experiment was implemented in a mixed-device survey with 3,374 German respondents from a non-probability web panel in 2018. The survey addressed a variety of topics, including consent to data linkage. Respondents were randomly assigned to either the PC or mobile group. In this experiment, we manipulated the question order and screen presentation of closed and open-ended feedback questions. The analysis focuses on a comparison between mobile and PC respondents. The indicators of response quality used in the analysis are item-nonresponse, response time, number of themes mentioned, and content (issues mentioned).

Results: Question order effects and important response quality differences between PC and mobile users have been found. Whereas the closed feedback format clearly reduced nonresponse and increased the number of mentioned topics, the open-ended format provided unique insights into respondents’ opinions toward consent requests to data linkage which were not detected by the closed format.

Added Value: This presentation provides valuables insights into the optimal implementation of feedback questions in mixed device studies.

List-style open-ended questions in Web surveys: A comparison of three visual layouts

Tanja Kunz1, Katharina Meitinger2

1GESIS Leibniz Institute for the Social Sciences, Germany; 2Utrecht University, the Netherlands

Relevance & Research Question: Previous studies on the visual layout of open-ended questions in web surveys have shown that respondents are responsive to verbal and visual design variations of the answer boxes. Findings consistently showed that several list-style answer boxes as compared to one large answer box elicit more elaborated answers. By contrast, a dynamically growing number of list-style answer boxes where respondents initially are exposed to one fixed answer box and further answer boxes are displayed only after they have clicked in the previous one seems to be less effective. The use of follow-up probes, in turn, where respondents are asked to provide further information on the following screen increases the number of themes. Despite a higher response elaboration with all these methods, item nonresponse remains an important issue. This paper aims at comparing all three methods in order to identify the optimal design which optimizes respondents’ answers to list-style open-ended questions without increasing item nonresponse.

Methods & Data: In two experiments embedded in a web survey on “Politics and Voting Behavior”, respondents from a nonprobability online panel (N=4,371) in Germany were randomly assigned to one of three experimental conditions, namely (a) a static design with six fixed list-style answer boxes, (b) a dynamic design with up to six list-style answer boxes displayed one after the other depending on whether respondents clicked on the previous one, and (c) a follow-up probe design providing three fixed list-style answer boxes on the initial screen and additional three on the next screen. The open-ended questions were on “satisfaction with democracy” and “current problems in Germany”, respectively.

Results: Findings showed that especially the follow-up probe design yielded more elaborated answers in terms of the number of characters and the number of themes mentioned, whereas a dynamic design was least effective. Overall, no differences in item nonresponse were found between the three visual layouts.

Added Value: The study contributes to the systematic assessment of visual design variations in open-ended questions to identify the optimal design with respect to data quality in list-style open-ended questions which are implemented in web surveys.

Kunz-List-style open-ended questions in Web surveys-151.pdf
3:30 - 4:30B11: Online Reputation and Influencer Marketing
Session Chair: Christian Kämper, Interrogare GmbH, Germany
Room 158 

The Reputation Effects in C2C Online Markets: A Meta-analysis

Ruohuang Jiao, Wojtek Przepiorka, Vincent Buskens

Utrecht University, Netherlands, The

Relevance & Research Question: We use the reputation effect on selling performance as an indicator of the effectiveness of reputation systems in C2C online markets. The purpose of reputation systems is to solve trust problems and promote cooperation among buyers and sellers. The larger buyers’ needs for information about sellers’ trustworthiness, the larger will be the correlation between seller reputation and selling performance. Consequently, the more effective a reputation system is at screening untrustworthy sellers, the smaller will be the reputation effect.

Methods & Data: We integrate the results from 96 existing empirical studies with a meta-analytic method to investigate the existence of reputation effects. We compare the correlational effects estimated between three types of seller reputation variables (number of positive and negative ratings, as well as feedback score) and four types of selling performance variables (probability of sale, selling price, selling quantity and the ratio of selling price to reference price). Because of the differences in analysis methods across the 96 studies, we use conversion techniques to calculate comparable effect sizes.

Results: Our meta-analysis confirms the general existence of the reputation effect. Especially the number of positive ratings exhibits a consistent, significantly positive effect on all types of selling performance with correlational effect sizes ranging from 0.06 to 0.27. The feedback score (i.e. the number of positive minus the number of negative ratings) also has a positive effect on selling performance, but is generally smaller and ranges from 0.03 to 0.09. Results regarding the number of negative feedbacks are mixed although the effects are, as expected, generally negative.

Added Value: Our study corroborates the existence of the reputation effect across different operationalizations of seller reputation and selling performance. It also shows however that the range of the effect is considerable, which could indicate that in some C2C markets reputation systems are more effective at screening untrustworthy sellers and thus reduce the information demand on the part of the buyers. In a next step, we will meta-analyze these reputation effects to obtain a better understanding of the conditions under which effect sizes vary.

Jiao-The Reputation Effects in C2C Online Markets-212.pdf

Is influencer marketing overpromising?

François Erner2, Jonathan Heinemann1

1respondi, Germany; 2respondi, France

Relevance & Research Question

For the brands, influencers offer new opportunities in digital marketing. They might be a more intimate, unformal, and therefore more convincing touchpoint with consumers. But before decisions can be made some questions need to be addressed. Is it worth it for a brand to sponsor youtubers? Who is the real audience of youtubers ? What is their “true” influence?

Methods & Data

We made a selection of the TOP 50 most influential German youtubers among various industries (fashion and beauty, food, hi-tech, tv show and movies, gaming).

We have a panel rolling out in Germany of # 2000 nationally representative people who have agreed to share their navigation data (on computer and / or mobile devices) with us. They have installed a tracking software on one or several devices. As such we are able to detect who watched a youtube video from an influencer (roughly 25 % of our sample had seen at least one video from the TOP 50 influencers), and what they did before and after.

Influencers post some videos sponsored by brands, websites. With our methods, we can measure the impact of a sponsored content; to what extent it brings traffic to the sponsor’s website.

We also run a survey among the identified audience of these influencers in order to understand their motivations.


The ROI of influencers strongly depends on the target and the industry. Youtubers have far more impact among young people. Cosmetics influencers are far more influential than food influencers. We have measures which prove that an important share of young people interested in cosmetics tend to use influencers as a reliable source of information and to follow their recommendations. We are also able to explain the logics of influence: What differentiates a good influencer from a “bad”one?

Added Value

For each brand we are able to build an index of performance for each influencer. With this method a brand is able to decide which influencer is the most impactful.

AI Pack Screening Model - Applying Data, Expertise & Artificial Intelligence to Screen Packaging Concepts

Christian Dössel, Hervé Turpault

PRS IN VIVO Germany GmbH, Germany

Relevance & Research Question:

In a world of insights that is rapidly changing due to new thinking (especially behavioral science), new data streams and new analytics and an increased time and budget pressure, we have decided to pursue a new path for screening packaging designs.

Methods & Data:

The AI Pack Screening Model is based on human expertise, data mining and artificial intelligence. Combining these three forces we have come to a predictive model of future success of pack designs without conducting surveys.

The "heart" of the Model consists of our own database with more than 25.000 studies on packaging research conducted over the past decades. It is trained by the input of variables that are validated to in-market Sales Performance, relationship with and across key metrics of pack designs and the impact of different market scenarios (e.g re-stage vs. new product, small vs large share of shelf, well established brands vs. new brands, etc.).

These Learnings have been applied to build a model for the screening process of pack designs which has been tested against survey data and which is continuously learning from new cases.

In addition to applying the model based on information by database learning to new designs, Human Experts are needed to provide a holistic perspective and to integrate dimensions/strategies specific to each brand’s situation. Those experts are Senior PRS IN VIVO professionals who will be activated at the beginning of the process.


The presentation will cover the main principles of the process as well as examples of outputs and will talk about first hand experience of combining human expert evaluation with AI based algorithms.

Added Value:

Thanks to the combination of Human Experts and AI results can be delivered within one week, catering the demand for faster turn around of research results in a rapidly moving insight world.

3:30 - 4:30C11: Mixed-Modes and Mixed-Devices
Session Chair: Tobias Rettig, University of Mannheim, Germany
Room 149 

Coverage Error in Smartphone Surveys Across European Countries

Tobias Baier, Anke Metzler, Marek Fuchs

Darmstadt University of Technology, Germany

Relevance & Research Question:

Surveys conducted solely in the mobile Web using smartphones as the only user device (mobile Web only surveys) provide several advantages as compared to traditional non-mobile Internet or mixed-device Web surveys. They potentially offer the use of samples consisting of randomly generated mobile phone numbers and text message invitations whereas traditional online surveys relying on e-mail invitations do not allow this procedure. Furthermore, smartphones provide paradata and sensor data for passive data collection. However, coverage error is a major threat to smartphone surveys and their potential benefits as mobile Web penetration considerably differs between socio-demographic groups (Keusch et al. 2018; An-toun 2015; Couper et al. 2015; Fuchs & Busse 2009) and across countries (Fuchs & Metzler 2014). This paper aims to estimate coverage error concerning country-specific differences in their development over time with respect to smartphone penetration.

Methods & Data:

Eurobarometer data from 2014 to 2018 across 28 European countries are used to estimate the relative coverage bias using six socio-demographic variables. Bias estimates are then analyzed over time and across countries with respect to country-specific smartphone penetration rates.


All countries assessed experienced a growth in smartphone penetration. Overall, coverage bias is declining over time and approaches the coverage bias of surveys in the non-mobile, landline Internet. Some countries already feature overall lower biases for mobile Web. However, country-level analysis shows that there are both countries with decrease as well as in-crease in coverage bias. Accordingly, growth in smartphone penetration does not predict the development of coverage bias.

Added Value:

The results shall inform which European countries are more or less suitable for the use of mobile Web surveys in the general population and how these differences relate to contextual conditions.

Baier-Coverage Error in Smartphone Surveys Across European Countries-140.pdf

Data quality in mixed-mode mixed-device general population UK social survey: Evidence from the Understanding Society Wave 8

Olga Maslovskaya, Gabriele Durrant, Peter WF Smith

University of Southampton, United Kingdom

Relevance & Research Question: We live in a digital age with high level of use of technologies. Surveys have started adopting technologies including smartphones for data collection. There is a move towards online data collection in the UK, including an ambition to collect 75% of household responses online in the UK 2021 Census. However, more evidence is needed to demonstrate that the online data collection will work in the UK and to understand how to make it work effectively. This paper uses the first available in the UK large scale mixed-mode and mixed-device social survey Understanding Society Wave 8 where 40% of the sample were assigned to online mode of data collection. It will allow comparison of data quality between face-to-face and online modes of data collection as well as between different devices within the online mode. This analysis is very timely and will fill this gap in knowledge.

Methods & Data: This analysis uses the main survey of the Understanding Society Wave 8. Descriptive analysis and then linear, logistic or multinomial logistic regressions are used depending on the outcome variables to study data quality indicators associated with different modes first and then with different devices in the online part of the survey. The following data quality indicators will be assessed: break-off rates, item nonresponse, response style indicators, response latencies and consent to data linkage.

Results: The detailed results will be available in mid-January 2019. Comparisons to results from the Understanding Society Innovation Panel and to results from other countries will be drawn.

Added Value: The originality of the analysis lies in addressing the underresearched area of data quality issues associated with different devices in mixed-mode and mixed-device surveys in the UK. The findings from this analysis will be instrumental to better understanding of data quality issues associated with mixed-mode and mixed-device surveys more generally and, specifically, in informing best practice for the next UK Census 2021. The results can help improving the design of the surveys and response rates as well as reducing survey costs and efforts.

Survey recruitment in 160 characters: Composition and Quality of a new mobile sampling strategy

Hannah Bucher, Matthias Sand

GESIS, Germany

Relevance & Research Question:

The strongly increasing number of people who own a mobile device with internet access implements new developments into web-survey-research. This increasing rate also has an impact on the devices used to complete a web-survey. Further, some studies investigate differences in demographics of the respondents associated with the used device, therefore it seems interesting to examine the possibilities of recruiting survey units via mobile phones.

Methods & Data:

As previous studies either used mobile phones as a separate sampling frame for CATI-surveys or used text messages to invite participants of an access panel to take part in specific web-surveys, we combined these approaches and investigate a new survey-sampling approach for web-surveys: The recruitment of people for a Web-Survey via a mobile RDD-mobile phone sampling with an invitation via text message.


Preliminary results indicate that this sampling method is not very suitable for questioning the broad population, due to problems occurring at different levels:

On the one hand, considerable technical problems arose in the process of verifying automatically generated mobile phone numbers (HRL-lookup). On the other hand, less than one percent of the sent text messages lead to an actual interview.

Added value:

Our research is the first to investigate a new probabilistic recruitment strategy for mobile web surveys. Therefore, it contributes substantively to the exploration of new survey recruitment strategies to replace or complement existing recruitment strategies that increased in costs such as face to face surveys or decrease in quality such as telephone surveys.

3:30 - 4:30D11: Alumni Get-Together des Masterstudiengangs Markt- und Medienforschung

Eine Veranstaltung des Studiengangs Markt- und Medienforschung der TH Köln

Das Get-Together ist offen für alle Konferenzbesucher.

Room 400 

Contact and Legal Notice · Contact Address:
Privacy Statement · Conference: GOR 19
Conference Software - ConfTool Pro 2.6.118
© 2001 - 2018 by Dr. H. Weinreich, Hamburg, Germany