Measuring the Unmeasurable: Two Cohorts, Two Methods, Four Results, Six Permutations
Relevance & Research Question:
Measuring satisfaction is difficult, be it customer or political satisfaction, for many reasons. Satisfied people often keep quiet, dissatisfied ones are more likely to speak up or complain. Satisfaction is difficult to quantify, it is unclear if questionnaire surveys reflect absolute satisfaction correctly or not. Last, the “Satisfaction Paradox”: averaging detailed satisfaction scores yields a worse score than asking for overall satisfaction.
Methods & Data:
In recent years, the prediction market method has expanded its usefulness beyond its origin, predicting election outcomes. Researching new product ideas or concepts, optimised pricing, customer and political satisfaction are now emerging as promising applications.
In a recent project, we found indications that the Gold Standard of satisfaction research, a questionnaire presented to a random customer sample may give inferior results compared to a self-selected crowd and a prediction market, considering the underlying purpose of such research.
We will present a case study about citizens’ political satisfaction, comparing the results of Prediki PROMPT, a new quali-quant method based on advanced prediction market technology, to traditional questionnaire results. Our case is based on two cohorts - a representative n=1000 and a self-selected of n=1500 - each doing both exercises which produces four data series and six relative comparisons.
These combinations not only shed a light on citizen satisfaction (or lack thereof) with Austria’s central government. We will show how the four results compare. Differences point to System 1 vs. System 2 responses, Relative errors indicate that an absolute measure of satisfaction is in fact possible, however that the current Gold Standard is not it. We will present how crowdsourcing yields a more authentic interpretation of these results, for more insight into why satisfaction levels are as they are.
A better read on customer satisfaction will yield significant financial and non-financial benefits for clients and governments alike. It will increase customer loyalty, thus secure more repeat business. It will focus businesses and organisations on the right actions to increase customer satisfaction while saving money and time on measures which do not.