Methodology
What we provide, what a row looks like, how the data is collected and how it is delivered.
How the indices are built
The UK and AU Online Shelf-Price Indices follow the posted online shelf prices of the same products from one week to the next and chain the weekly changes from a base of 100. The UK index page sets out its method: UK Online Shelf-Price Index; the AU Online Shelf-Price Index follows from 19 October 2026, with its own method page. City Hotel Prices, from after week 42, follows the change in the prices of the same (matched) hotels from week to week, chained from a base of 100 at ISO week 2026-W40.
What we provide
Shelf prices, promotions and, where a retailer publishes one, loyalty prices for every product the major supermarkets list online, joined across retailers by EAN barcode where the retailer publishes one. A weekly full-catalogue series for the UK and Australia, and a daily fixed basket of 212 UK staples for consistent day-to-day tracking. Retail-level price data with this structure is a nowcast of food inflation weeks ahead of official statistics, and a competitor-price series for CPG and retail analytics.
UK weekly catalogue
LiveUK daily basket
LiveAustralia weekly catalogue
October 2026UK property listings
Actor liveGoogle Hotels rates
LiveUK open-data lookups
From 15 Oct- Public contractsfrom 15 Oct
- Address intelligencefrom 15 Oct
- LEI lookupfrom 16 Oct
Compare the series in a table
| Series | Coverage | Frequency | Rows per period | History from |
|---|---|---|---|---|
| UK weekly catalogue | Tesco, Sainsbury's, Asda, Aldi; Morrisons in the live Actor now (opt-in), and in the weekly series from November 2026 | Weekly, Sunday | ~73,000 | 27 Sep 2026 |
| UK daily basket | 212 staple products across the four retailers (up to three per retailer for each of 20 staples), the same items every day. | Daily | 212 | 24 Sep 2026 |
| Australia weekly catalogue | Coles, Woolworths, Aldi Australia | Weekly | ~65,000 | October 2026 |
| UK property listings (Actor live; weekly series planned) | Listing counts, asking prices and days on market by area from Rightmove, Zoopla and OnTheMarket, with planning data | Weekly (planned) | — | Q4 2026 |
| UK open-data lookups (Actors) | UK Public Contracts Lookup, Address Intelligence, LEI Lookup | On demand | — | October 2026 |
| Google Hotels rates (Actor) | Any destination Google Hotels covers, by city search or hotel id; search results and per-booking-site vendor rates | On demand | — | September 2026 |
Weekly UK Online Shelf-Price Index: https://data.yappman.com/shelf-price-index
Weekly AU Online Shelf-Price Index: from 19 October 2026. City hotel price index: after week 42.
History accumulates weekly from the dates shown and is never revised. Trials are offered from three months of history.
What a row looks like
One row per product per retailer per period. Four real rows from the free daily-basket sample, scraped on 8 October 2026.
| Retailer | Product | EAN | Shelf price | Was price | Loyalty price |
|---|---|---|---|---|---|
| Tesco | Wildfarmed Sliced Seeded Bread 550g | 5065009224302 | £2.25 | — | £1.95 |
| Sainsbury's | Hovis Soft Medium Sliced White Bread 800g | 5010003000131 | £1.25 | — | £1.10 |
| Asda | High Protein Half Fat Mature Cheddar Cheese 300g | 5000295163522 | £2.00 | £3.42 | — |
| Aldi | Medium Sliced White Bread | — | £0.55 | — | — |
Fields Supermarket data
One row per product per retailer per period: retailer, retailer product id, EAN, name, brand, pack size, shelf price, was-price, unit price and measure, loyalty price, promotion text and end date, stock flag, category path, URL, image URL; rating and review count where the retailer shows them; timestamp; and, in the Australian series, currency, market and store id. Loyalty price where the retailer publishes one: Tesco Clubcard, Sainsbury's Nectar and Morrisons More Card. Asda, Aldi, Coles, Woolworths and Aldi Australia publish no loyalty price, so the field is empty for them. Shelf price is always the price a non-member pays; loyalty pricing is never folded into it.
How it is collected Supermarket data
Source. The retailers' own public websites, read logged out. No accounts, no baskets, no personal data of any kind.
Method. Automated reads of the pages and JSON any visitor receives. Requests are made one at a time, at no more than two a second per retailer (one a second for Morrisons); in practice about 0.6 to 1.5 a second on average. Nothing a retailer's published robots.txt disallows is ever requested; paths were checked against each source's robots.txt where one could be read.
Quality. Daily canary runs for every retailer, an automated weekly health review of every canary, snapshot and index build, and email alerts on failed runs. Weekly reconciliation of each retailer's product count against the previous week and the first full week; a drop of more than 10% is flagged. Parsers that fail loudly rather than emit partial rows; missed periods recorded as gaps, never interpolated.
Change control. A change log ships with every delivery; schema and coverage changes are announced 30 days ahead, under Notices.
Delivery Supermarket data
Weekly (and, for the basket, daily) CSV files, one per retailer per period, as a download. Parquet, and delivery to your S3 bucket, SFTP or a signed download link, are available on request. The live data is also available programmatically through the Apify API. A basket-pricing endpoint (price a list of barcodes across all retailers in one call) is in preparation. Weekly data is available by 08:00 UTC on Sunday; the daily basket by 07:00 UTC.
Due diligence Supermarket data
A methodology and due-diligence document is available on request. It covers provenance, collection method, robots.txt compliance, the terms-of-service position, database-right and copyright treatment, personal data (none), anti-bot handling, audit fields, coverage limits, accuracy checks, retention and corporate details. Retailer names are used only to describe the source of public product information; no affiliation, endorsement or partnership is claimed.