Scanner data
Scanner data are digital transaction data recorded at the cash registers of retail shops which inform about turnover, sales and the types of items sold. Scanner data have the potential to digitise various sets of statistics and to assure and improve data quality. Scanner data can be used in consumer price statistics or to determine regional price differences, for example.
Scanner data in consumer price statistics
Digital point-of-sale (POS) data of the retail sector, also referred to as scanner data, are a promising data source for price statistics. In official consumer price statistics, manual price collection by price collectors in shops or service enterprises has been supplemented, or in part replaced, by new forms of price collection. These include in particular automated price collection on the internet (web scraping) and the use of scanner data. In contrast to web scraping, scanner data have not yet been used in official German consumer price statistics.
It is not information about individual transactions that is relevant for price statistics but aggregated sales data for individual items sold by large retail chains or shops with high turnover. Overall, scanner data have the potential to assure the accuracy and quality of consumer price measurement in Germany for selected product groups in view of dynamic pricing and gradually replace traditional price collection in shops while reducing the related costs and effort in the long run.
The use of scanner data makes it possible to record the prices charged and the quantities sold in the respective shops for a much longer period than is currently the case. Generally, prices have been collected in a specific shop on one day per month. The increasing number of observations improves the accuracy and enables additional evaluations to be made, for example regarding the price development of organic products.
Compared with the current approach to calculating the consumer price index (CPI), however, the use of scanner data presents completely new challenges of a methodological, organisational and technical nature. The Federal Statistical Office (Destatis) is currently investigating the form in which scanner data can be used in the ongoing statistics production. Findings about the current buying behaviour are published in our press releases. Detailed information on the methodology and the character of the data can be found in the following articles of our scientific journal WISTA
Edition 4/2020: Einsatz von Scannerdaten während der COVID-19-Pandemie
Edition 4/2019: Fortschritte beim Einsatz von Web Scraping in der amtlichen Verbraucherpreisstatistik
Edition 2/2019: Nutzung von Scannerdaten in der Preisstatistik
Use of scanner data to determine regional price differences
Scanner data can be used not only in consumer price statistics but also in various other statistics, for instance to determine regional price differences. The Federal Statistical Office conducted two feasibility studies on food, health, beauty and toiletry products to analyse the suitability of scanner data for reflecting regional price differences.
In the second feasibility study the Federal Statistical Office used scanner data of various discount stores and shops specialising in health, beauty and toiletry products from the year 2020. The individual products sold by all retail chains were aggregated at the level of administrative districts and towns not attached to an administrative district. An annual average price was thus obtained for each product per district or town. For the complete range of food offered, differences of approximately four percentage points were determined in regional pricing by means of the multilateral GEKS Tornqvist price index. The differences are larger in some cases when individual product groups are considered separately. The following interactive maps show the price indices for the entire range of food offered and for individual product groups.
In the first feasibility study the Federal Statistical Office used scanner data of a food retail chain from the year 2017. The data from the shops of that chain were aggregated on the level of districts and towns not attached to an administrative district. For the complete range of food offered, differences of somewhat more than three percentage points were determined in regional pricing by means of the multilateral GEKS Fisher price index. These differences are much larger in some cases when individual product groups are considered separately. The following interactive maps show the price indices for the entire range of food offered and for products sold over the counter.
The results of the study and further analyses on how to ascertain the determinants of regional price differences as well as an application regarding income inequality were presented at the Statistical Week 2021 in Kiel (Book of Abstracts) and at the virtual 63rd ISI World Statistics Congress 2021 (Proceedings / Video).