Non-Domestic Energy Performance Certificate (EPC) Data
Published by Ministry of Housing, Communities and Local Government
About this dataset
This page explains how the EPC (Energy Performance Certificate) dataset published on the City Observatory is kept up to date. Where this runs Workspace: BCC City Observatory - DATAFLOW Gen2 Lakehouse: CityObservatoryBronze Notebook: EPC Download Bronze and upload to Observatory Schedule: Runs automatically with a schedule directly in the Notebook What the notebook does Download — calls the MHCLG (Ministry of Housing, Communities and Local Government) Energy Performance Data API, which returns a zip file containing certificate data for the whole of England and Wales, split across many years' worth of files. Transform — unzips the file in memory, keeps only rows where the local authority is Birmingham, then removes duplicate certificates for the same property, keeping only the most recently lodged one per property (matched on UPRN). Upload — the finished CSVs (one for domestic properties, one for non-domestic) are sent to OpenDataSoft via FTP, which is what powers the published dataset on the City Observatory. The same CSVs are also saved into the Lakehouse (see below). Where the output lands Each run produces output in two places: Location What's there Purpose CityObservatoryBronze Lakehouse > Files > EPC folder The two staged CSV files, same content as what's uploaded to FTP A working copy used by the notebook, and a point-in-time snapshot of what was last sent CityObservatoryBronze Lakehouse > Tables > EPC schema The same data as queryable Delta tables Lets anyone in Fabric query, check, or build on the data without needing FTP or API access Table locations: Domestic certificates: EPC.Domestic Certificates Raw Non-domestic certificates: EPC.Non Domestic Certificates Raw Where it ends up on the FTP Dataset FTP folder Filename Domestic certificates /private/epc EPCDomesticBirminghamNotebook.csv Non-domestic certificates /private/epc EPCNonDomesticBirminghamNotebook.csv The FTP server is OpenDataSoft's own ( eu.ftp.opendatasoft.com ), and the files landing here are what OpenDataSoft picks up to refresh the published dataset on the City Observatory. If something needs fixing Open the EPC Download Bronze and upload to Observatory notebook in the BCC City Observatory - DATAFLOW Gen2 workspace. Check Recent runs for the notebook, or the associated Data pipeline's run history in the Monitor hub, to see if the last scheduled run succeeded or failed. If it failed, open the run and check the cell output for an error message. Common causes are the MHCLG API being temporarily unavailable, a change in the source file's format, or an expired credential. Credentials (the MHCLG API bearer token and the FTP password) are set as plain variables near the top of the notebook. If either has changed, update it there. You can re-run the notebook manually at any time by opening it and selecting Run all . This re-downloads, filters, and re-uploads the data immediately. If the Lakehouse tables look wrong or missing, check under Tables that the EPC schema still exists, since the notebook will fail to write if it's been renamed or removed.
Live preview
The first rows, fetched from Opendatasoft hub when you open this page.
Loading the first rows from the source…