Overview and details of the sessions of this conference. Please select a date or location to show only sessions at that day or location. Please select a single session for detailed view (with abstracts and downloads if available).
C6: Mobile Web Surveys: Device Effects
Smartphones as digital companions
1Julius-Maximilians-University of Wuerzburg, Germany; 2Nottingham Trent University, UK
Relevance & Research Question: Our online study addressed the personal importance of smartphones for their users. Our assumption is that smartphones are omnipresent companions in everyday life and have by far transcended their objective status as technology. Users, as a result, have developed a rather emotional relationship, e.g. feelings of closeness, with their phones. Thus, we investigated the "emotional relevance" of participants’ phones in comparison to other electronic devices and to human beings.
Methods & Data: We developed an online instrument based on techniques from systemic psychotherapy. The original technique involves the positioning of pieces representing family members on a chessboard to visualize relational structures within a family (Gehring & Arnone-Reitzle, 1998). We replaced family members with (1) categories of people and (2) media devices, both pretested to be relevant. Consequently, our tool visualizes the relative importance of both humans and technology via the distances to the piece representing oneself. In addition, self-report scales measured the emotional relationship to the smartphone, levels of smartphone usage, phone-related trust, phone-related stress and stress relief (e.g. Aron, Aron & Smollan, 1996; Rempel, Holmes & Zanna, 1985; Satow, 2012). The full survey, including tool and scales, was conducted online in Germany and the UK (total n = 1168).
Results: Due to space restrictions only core results showing smartphones to be psychologically important beyond their mere technical functionalities can be summarised here. First, the smartphone was the fifth nearest piece on the board, thus revealing it to be (1) the most important technological device and even (2) more important than some categories of relevant persons (e.g. roommate, classmate). Second, survey measures revealed a range of significant predictors for psychological aspects of phone usage, in particular usage intensity (light, medium, heavy), operating system (iOs vs. Android) and closeness to phone as indicated by the position on the chess board.
Added Value: We take our findings as first evidence for our core assumption: smartphone users have on average established significant emotional relationships with their devices. The idea of smartphones as “digital companions” rather than mere digital devices is discussed against (media)psychological and evolutionary explanations.
Device effects on behaviour and participation in mobile-optimised online diaries
Schmiedl Marktforschung GmbH, Germany
Relevance & Research Question:
What demographic and behavioural factors differentiate users of smartphones, tablets and desktops, and what are their effects on diary participation?
Methods & Data:
800 active pool members in 6 countries each (Australia, France, Germany, Mexico, Spain, UK) - recruited via a 15-minute online screening questionnaire - were asked to fill out at least 3 diaries per week over 25-35 weeks between November 2014 and December 2015 on a fully mobile-optimised link.
There was an option for respondents to indicate that they had nothing new to report on, which was counted as unqualified diaries. Over the course of the study, 477.330 qualified diaries were collected. One qualified diary took 3-5 minutes.
- Smartphone users are somewhat more likely to report qualified diaries than desktop users.
- Smartphone users reported more qualified diaries in a shorter tenure in the study than desktop users. However participants show a high variance in their individual reporting behaviour.
- Device usage is differentiated by demographic factors (age, gender, country, income), but these did little to explain activity and tenure in the survey.
- Activity in social networks was only marginally relevant in explaining activity and tenure.
- Respondents who used a mix of devices to participate in the diaries stayed active longer and delivered significantly more qualified diaries than single device participants.
- There is an important device effect on the content participants reported on. Some content is more likely to be reported on a smartphone, while other content is more likely to be reported on a tablet or desktop.
The technical setup that allows completion on every device is a much more decisive factor in ensuring high activity and long tenure than respondent selection based on social demographics. Nevertheless, no single device user can be considered more valuable in terms of activity or tenure in online diary studies.
Different target groups use different devices, therefore it is necessary to offer online surveys fully optimized for smartphones, tablet and desktops alike. Also, different content is reported on each device, and therefore limitations not to provide a fully mobile-optimised diary will have effects on the results.
The Role Played by the Device Screen Size and by the Questionnaire Optimization within the Mobile Survey Participation
1University of Bergamo, Italy; 2RECSM – Universitat Pompeu Fabra, Spain
Relevance & Research Question: The mobile survey participation is currently a relevant phenomenon, worldwide. Mobile devices have different characteristics in comparison to traditional fixed-PCs or laptops. A key difference is the reduced size of the devices and of their screens. The enhanced portability is counterbalanced by some potential risks. Can the screen size affect the quality and the comparability of collected data? And which is the role that the questionnaire optimization can play?
Methods & Data: Data were collected in 2015, using the Netquest opt-in online panel for Spain, through a two-wave survey proposing twice the same questionnaire to the same respondents (1,608 panelist). Each panelist was randomly assigned, within a specific wave, to a survey condition (i.e. the device used and the questionnaire optimization, for participants using a mobile device). This research focuses only on mobile device respondents. In order to test our hypotheses, we computed a series of indicators -some based on paradata- related to both the participation process (e.g. the completion time, the answer consistency between waves, etc.) and the survey experience evaluation (perceived difficulty, appreciation, etc.).
Results: The device screen size usually does not significantly affect some indicators linked to the data quality/comparability, but can significantly affect variables (such as the survey length or the evaluation of the survey experience) that can determine the respondents’ burden and their willingness in participating again in surveys. However, most of the potential issues linked to the use of small sized devices can be attenuated proposing a questionnaire optimized for the mobile participation. The paper also investigates the interaction between these two variables and the consequences of re-proposing the same survey.
Added Value: Despite the widely acknowledged importance of the screen size, only few studies directly analyzed this key variable. Most of them just considered broader and nonhomogeneous categories of devices (i.e. smartphones and tablets) or used a limited set of indicators (mostly completion time). This work aims at using a wider range of indicators and at providing an updated and more precise view, taking into account the exact screen size (in inches) and its interaction with the optimization.
Focus on mobile surveys: Do the number of scale points and scale order affect rating scale results?
BiTS (Business and Information Technology School) / exeo Strategic Consulting, Germany
1. Relevance & Research Question:
Scale types (straight or odd scale points) and scale order (i.e. the order in which response options of a rating scale are presented) were frequently investigated in the past, since the question arose whether the results of surveys are influenced by the nature and application of a scale. However, experimental studies have not yielded clear and robust, but rather contradictory and unambiguous results. Considering a strong trend towards online surveys in general and mobile research in particular, this question appears in a new light (reduced interview time and visibility).
2. Methods & Data:
In order to analyze the influence on rating scales both mentioned factors, scale type as well as scale order an experimental online survey design was created (n=600, Germany, 16+ years), covering a 5 point vs. 6 point satisfaction scale, each with two different orders (positive-negative vs. negative-positive scale; 2*2 design). Subjects were asked to evaluate 5 facets of the interview (topic of the interview etc.). Four test groups were randomly generated, each receiving one specific scale form.
Based on the experimental setup, no significant differences between a 6-point and a 5-point scale can be identified as long as the scale direction is positive to negative, i.e. the best rating ("very satisfied") is shown left next to the statement/item. Particularly strong influences on the results are apparent when using a in the 6-point scale and changing the scale direction to a negative to positive order. In this case the evaluation of the interview is significantly worse than in test groups with a positive-negative ordered scale. Furthermore, this effect is particularly strong in a subgroup of respondents, which used a smartphone to participate in the web survey.
4. Added Value:
Firstly, it is irritating that the mean values on a 6-scale scale differ by more than 0.5 points depending on scale order. Secondly, it is disturbing that the effects increase when respondent used a smartphone. Thirdly, this has strong implications for developing a suitable questionnaire design for online / mobile studies.
Contact and Legal Notice · Contact Address:
Conference: GOR 17
|Conference Software -
ConfTool Pro 2.6.96
© 2001 - 2016 by H. Weinreich, Hamburg, Germany