Weather ELT with dbt
Planned
Overview
A planned ELT pipeline that loads daily forecasts from the free Open-Meteo API into Snowflake and transforms them with tested dbt models.
Problem statement
Weather data arrives as nested JSON from an API rather than as files. This project practises the ELT pattern: land the raw responses first, then model them in the warehouse with SQL that is versioned and tested.
Architecture
Planned design: a Python job calls the Open-Meteo forecast API for a set of UK cities and loads the raw JSON into a Snowflake staging table; dbt models flatten it into hourly and daily tables, with tests for nulls, ranges and duplicate readings.
Technologies
Python
Snowflake
dbt
- SQL