UK fuel price tracker

A historical UK fuel price tracker built on the GOV.UK Fuel Finder API. Prices are checked every 30 minutes and updates are stored to enable analysis the live API can't provide.

• Live dashboard — current averaged prices by region, brand and forecourt category
• Price history — trends over time for individual stations or filtered groups
• Interactive map — colour-coded price distribution across the UK
• Station search — filter by brand, location, postcode area, rural/urban classification and more
• Anomaly & outlier detection — suspicious or statistically extreme prices flagged and excluded from averages
• Price correction tools — editors can review and correct misreported values (editor/admin only)
• Data correction tools — brand normalisation, postcode overrides and coordinate fixes for source data issues (editor/admin only)
• REST API — all data available programmatically; see the API reference

How it works

This tool gathers data from the government's consumer fuel price finder — and stores it in a PostgreSQL database to build a historical price record that the API itself doesn't provide.

The GOV.UK API only serves live snapshots: current prices at the time of the request. There is no way to retrieve yesterday's prices or see how prices have changed over time. By scraping regularly and storing every price change, we build a time-series dataset that enables trend analysis, regional comparisons, and anomaly detection.

Data source

The Fuel Finder API is a government service that aggregates fuel prices reported by petrol station operators. It covers approximately 7,500 stations across England, Wales, Scotland and Northern Ireland, reporting prices for up to six fuel types.

The scraper

We use an API key granted by gov.uk to check developer.fuel-finder.service.gov.uk/public-api for updates to prices at UK fuel stations.

Fuel Finder API (GOV.UK)
    │
    │  OAuth2 client credentials → bearer token (1h TTL)
    │  GET /api/v1/pfs/fuel-prices?batch-number=N   (500 stations/batch, ~15 batches)
    │  GET /api/v1/pfs?batch-number=N               (station details)
    ▼
┌───────────────────────────────┐
│  Scraper                      │  Python — api_client.py + scrape.py
│                               │
│  1. Authenticate              │  OAuth2 client credentials
│  2. Fetch batches             │  Paginate through all stations + prices
│  3. Upsert stations           │  Insert/update station records
│  4. Insert prices             │  Append-only, deduplicated
│  5. Detect anomalies          │  Flag suspicious prices (not filter)
│  6. Refresh view              │  Rebuild current_prices snapshot
│  7. Enrich postcodes          │  Lookup new postcodes via postcodes.io
│  8. Backup to S3              │  Raw JSON (optional)
└───────────────┬───────────────┘
                │
                ▼
┌───────────────────────────────┐
│  PostgreSQL                   │
│                               │
│  stations                     │  ~7,500 fuel stations with lat/lng
│  fuel_prices                  │  append-only price change events
│  brand_aliases                │  raw → canonical brand mapping
│  brand_categories             │  brand → forecourt type
│  station_brand_overrides      │  per-station brand corrections
│  station_postcode_overrides   │  per-station postcode corrections
│  fuel_type_labels             │  fuel code → human name
│  postcode_regions             │  postcode → ONS region
│  postcode_lookups             │  postcodes.io enrichment cache
│  scrape_runs                  │  scrape execution history
│  current_prices               │  materialised view (live snapshot)
└───────────────────────────────┘

API usage guidelines

The scraper is designed to honour the GOV.UK Fuel Finder developer guidelines. The following safeguards are built in:

• Rate limiting — A client-side sliding window enforces the 30 requests-per-minute limit. If a scrape requires more than 30 API calls, the scraper waits until capacity is available rather than exceeding the quota.
• Retry with backoff — If the API returns a 429 Too Many Requests or a transient server error (500/502/503/504), the scraper retries with exponential backoff, respecting the Retry-After header when present.
• Compression — All requests include Accept-Encoding: gzip, deflate to reduce bandwidth, per the performance guidelines.
• No data redistribution — Raw API responses backed up to S3 are stored in a private bucket with all public access blocked. This data is retained solely for internal debugging, reprocessing, and data-integrity checks, and is never redistributed.
• Incremental fetching — Rather than re-fetching all prices every run, the scraper uses the effective-start-timestamp parameter to request only prices that have changed, minimising unnecessary API load.
• Sequential requests — Only one API request is in-flight at a time, respecting the one-concurrent-request-per-client limit.

Scrape modes

How prices are stored

The fuel_prices table is append-only, but with deduplication. When a scrape runs:

1. For each (station, fuel type) pair, the scraper looks up the most recently stored price
2. If the new price is the same as the stored price, it's skipped
3. If the price has changed, a new row is inserted with the current timestamp
4. The new price is checked against anomaly rules and flagged if suspicious

This means every row in fuel_prices represents a genuine price change event, keeping storage lean and making time-series analysis straightforward.

Anomaly detection

On insert, each price is checked against three rules. Suspicious prices are flagged as 'anomalous', but not ignored. On-site visualisations for average prices and trends over time exclude anomalous values to to avoid being skewed by obvious errors in the source data. However, the original reported values are preserved and are included in station price history tables and exports.

In practice the large_price_jump test can mean legitimate price reports are flagged when the previous price report was incorrect. If the price passes the first three tests, our tool should not automatically pass judgment on whether the old or the new value is correct. Such problems go away naturally from the dashboard when a station reports its prices accurately twice in a row.

Statistical outlier exclusion

Anomaly flags catch the most obvious errors, but the GOV.UK source data is manually entered by fuel station operators and inevitably contains subtler mistakes — for example, stations often misattribute B7 Standard Diesel and B7 Premium Diesel prices, or a stale or test value slips through. Even a few of these can skew averages noticeably.

Dashboard & current snapshot (Tukey IQR)

All averages shown on the dashboard cards, and the current-price breakdowns by region, brand, category, etc., exclude statistical outliers using the Tukey IQR (interquartile range) fence method — the same technique commonly used in box-plot charts:

1. For each fuel type, compute Q1 (25th percentile) and Q3 (75th percentile) of all current non-anomalous prices
2. Calculate IQR = Q3 − Q1 — the spread of the middle 50% of prices
3. Set fences at Q1 − 1.5 × IQR (lower) and Q3 + 1.5 × IQR (upper)
4. Any price outside these fences is marked as an outlier

The 1.5 × IQR multiplier is the standard Tukey threshold (introduced in John Tukey's Exploratory Data Analysis, 1977). It identifies values that are far enough from the central distribution to be suspect, without being so aggressive that it trims legitimate price variation.

The fences are recomputed each time the materialised view is refreshed (after every scrape), so they adapt automatically as prices change over time. Since IQR is applied to a single snapshot (the latest price per station), trending prices are not an issue.

Historical trend charts (Hampel filter)

For trend charts that show price history over weeks or months, a static IQR fence would be inappropriate — if prices trend upward steadily, earlier (legitimately lower) prices would be wrongly excluded as outliers. Instead, trend data uses a Hampel filter, a technique widely used in financial time series and signal processing:

1. For each time bucket (day or hour), compute the median and MAD (median absolute deviation) of prices in a sliding window around that bucket
2. If the bucket's average price deviates from the window median by more than 3 × MAD, it is replaced with the window median

This approach correctly handles trending data because each data point is only compared against its temporal neighbours, not the entire range. The window covers ±3 days for daily data and approximately ±1 day for hourly data.

Brand normalisation

The API provides brand names inconsistently — ESSO, Esso, esso all appear for the same company. Three layers of normalisation clean this up:

1. Brand aliases — bulk mapping of raw API strings to canonical names (e.g. "TESCO" → "Tesco"). Managed via the Data Cleanup tab.
2. Station overrides — per-station corrections for edge cases where the API brand is wrong or the alias isn't granular enough.
3. Resolution order — station_override > brand_alias > raw_brand_name. Overrides always win.

Raw brand values are never modified in the database. Normalisation is applied in the materialised view, so any change can be reversed.

Forecourt categories

Stations are classified into categories based on their canonical brand name via a lookup table, with the exception of those flagged as is_motorway_service_station in the raw data. This flag always takes priority over our lookup.

We do not honour the is_supermarket_service_station flag from the raw data because it has proved to be too unreliable. For example BP, Texaco, and Maxol forecourts are frequently flagged as supermarkets.

See also glossary: forecourt categories and the “Supermarket / Motorway” filters.

Regional mapping

Each station's postcode is mapped to an ONS-style region using the first 1–2 letters of the postcode (the postcode area). For example, SW1A → SW → London, M1 → M → North West.

This enables regional price comparisons (e.g. "London is 3p/litre more expensive than the North East") without requiring geocoding.

Postcodes.io enrichment

Each unique station postcode is looked up via postcodes.io, a free and open API for UK postcode data. The results are cached in a postcode_lookups table, providing:

• Authoritative coordinates — fixes ~85 stations where the Fuel Finder API reports incorrect lat/lng (e.g. sign errors placing stations in the North Sea)
• Administrative geography — local authority district, county, ward, parish
• Parliamentary constituency — for political analysis
• Rural/urban classification — ONS RUC 2021 (England/Wales) and Scottish Government 6-fold (Scotland) categories
• Statistical areas — LSOA, MSOA, built-up area

Postcodes that postcodes.io doesn't recognise are recorded (so they aren't retried) and flagged in the Data tab as potential data quality issues. Two correction tools are available for editors — see Data correction tools below.

Rural/urban classification

The UK does not have a single rural/urban classification — England/Wales and Scotland use separate systems with different categories. Postcodes.io returns the raw classification for the relevant country, so our database stores two distinct sets of labels:

The search and trends filters expose the full original labels from each system, so you can filter on exactly the category you want. The dashboard chart groups each system's categories separately — it does not attempt to equate them, since the two systems use different population thresholds, spatial units, and definitions. Colour coding indicates approximate analogies (red = urban, green = rural, blue = small towns) without asserting equivalence. The mapping is:

Within each system, the proximity dimension is collapsed on the dashboard: the England/Wales "nearer to" and "further from a major town or city" variants are merged, as are Scotland's "Accessible" and "Remote" variants. The full granularity remains available in the search and trends filters.

Licensing

Postcodes.io source code is available under the MIT Licence. The underlying postcode data is used under the following terms:

• Great Britain postcode data is used under the Open Government Licence
• Northern Ireland postcode data (BT prefix) is used under the ONSPD licence

Contains Ordnance Survey data © Crown copyright and database right 2026.
Contains Royal Mail data © Royal Mail copyright and database right 2026.
Contains National Statistics data © Crown copyright and database right 2026.
Contains NRS data © Crown copyright and database right 2026.

The materialised view

current_prices is a PostgreSQL materialised view that provides the latest price per station per fuel type. It joins together:

• The most recent price from fuel_prices
• Station details from stations
• Canonical brand name (via aliases and overrides)
• Forecourt type (via brand_categories)
• Region (via postcode_regions)
• Human-friendly fuel names (via fuel_type_labels)
• Authoritative coordinates, admin district, constituency, rural/urban (via postcode_lookups)
• Corrected postcodes (via station_postcode_overrides)

The view is refreshed after each scrape run and when you press Refresh View on the Data Cleanup tab. All dashboard, map, search and API queries read from this view for fast, consistent results.

Running on a schedule

For production use, the scraper is designed to run on AWS Lambda with EventBridge Scheduler:

• Every 30 minutes: incremental scrape (fetch changed prices only)
• Daily at 03:00 UTC: full scrape (refresh all station metadata + prices)

Raw JSON responses from each scrape are optionally backed up to S3 for audit and replay purposes.

Data correction tools

The GOV.UK Fuel Finder API data is manually entered by station operators and contains various quality issues beyond price errors. The Data tab provides tools for editors and admins to correct these without modifying original source data.

Brand normalisation

Inconsistent brand strings are cleaned up via brand aliases (bulk mapping) and station overrides (per-station corrections). See Brand normalisation above for details. Both are managed on the Data tab.

Postcode overrides

Some stations have mistyped, expired or otherwise incorrect postcodes. Since the postcode drives geographic enrichment (region, constituency, district, rural/urban classification), a bad postcode means the station is missing from geographic breakdowns or placed in the wrong area.

Postcode overrides let editors replace a station's postcode for enrichment purposes. The original postcode is preserved in the stations table. When an override is saved:

1. The corrected postcode is looked up via postcodes.io to populate full enrichment data
2. The override is stored in station_postcode_overrides
3. After a view refresh, the current_prices view uses the corrected postcode for all geographic joins

Overrides are managed on the Data tab → Postcode Overrides section, or directly from the Postcode Issues table via the "Fix postcode" button.

Coordinate fixes

When a postcode is not recognised by postcodes.io (and therefore has no authoritative coordinates), the station falls back to the API-reported latitude and longitude — which can be incorrect. Some stations report coordinates outside the UK entirely (e.g. sign errors placing them in the North Sea).

The Postcode Issues table highlights these stations and provides a "Fix coords" button to manually set latitude and longitude for the postcode. This updates the postcode_lookups table so the corrected coordinates are used in the materialised view.

In many cases, a postcode override is a better fix than a coordinate correction — if the postcode itself is wrong, overriding it also fixes the region, constituency and other geographic fields, not just the coordinates.

Price corrections

Misreported prices can be corrected via the price editor on the Anomalies tab. See How prices are stored for details on anomaly detection. Corrections are stored separately in price_corrections — original price records are never modified.

Glossary

Terms used throughout the dashboard that may not be immediately obvious.

Stations and brands

Station

A physical forecourt at a specific address. Each station has a unique node ID assigned by the Fuel Finder API (see below). A station's trading name is what it displays to the public — usually the brand name, but sometimes a locally branded variant (e.g. "Welcome Break Corley").

Brand

The operator or brand a station trades under — Shell, Tesco, BP, and so on. The API reports a raw brand string per station; normalisation maps these to canonical names (see Brand normalisation above). Multiple stations can share a brand; one brand maps to exactly one forecourt category.

Station node ID

The unique identifier for a specific station in the Fuel Finder API — a short alphanumeric string (e.g. A1B2C3). It is stable across scrapes and is used as the primary key throughout the database. You need a node ID to add a station-level brand override, or to look up a single station's price history via the API.

Forecourt categories and the “Supermarket / Motorway” filters

There are two separate ways to filter by supermarket or motorway, and they mean different things:

Category filter (the "Supermarket" and "Motorway Operator" / "Motorway" options in the Category multi-select)

These use the derived category — the forecourt type we assign based on the station's canonical brand name (see the category table above). "Supermarket" means Tesco, Asda, Sainsbury's, Morrisons, Waitrose; "Motorway" means any station whose brand is in the Motorway Operator category or has the motorway flag set.

"Supermarket only" and "Motorway only" checkboxes

These use the raw flags reported directly by the Fuel Finder API — is_supermarket_service_station and is_motorway_service_station. The API flags are set by the station operators themselves and are unreliable: some major oil company sites are flagged as supermarkets, and some genuine motorway sites are not flagged at all. Use the Category filter for the more accurate, curated classification; use these checkboxes when you specifically want to query what the source data reports.

Brand normalisation report

The normalisation report (Data tab → Report) shows how each raw brand string in the database has been resolved to a canonical name. The Method column and the filter dropdown use three terms:

Aliased

The raw brand string from the API matched a brand alias rule, so it was mapped to a canonical name (e.g. TESCO PFS → Tesco).

Overridden

This specific station has a station override that sets its canonical brand directly, regardless of the alias table. Overrides are used for one-off corrections — a station whose API brand is wrong, or where the network-wide alias isn't granular enough. Overrides always take priority over aliases.

Unmapped

The raw brand string has no alias and no override, and the brand has no entry in the category table. Consequently, it falls back to showing the raw string as its brand name and is classified as "Uncategorised". Unmapped brands are worth reviewing — they may be new entrants to the API, typo variants, or obscure regional operators that need an alias adding.

Country filter: "Other / Unknown"

The country field for each station comes from two sources: the Fuel Finder API itself (a free-text country string) and the postcodes.io enrichment lookup. Both are normalised to England, Scotland, Wales, or Northern Ireland.

"Other / Unknown" covers stations where the country cannot be determined — typically because the postcode wasn't recognised by postcodes.io and the API's own country string was blank, misspelled, or didn't match a known value. There are usually only a handful of these at any time; they often correspond to stations with postcode data quality issues.

Contact

To report problems, request API keys or ask questions, contact Luke Hoyland at luke@hoy.la.

Source code is available on GitHub.