Back to projects
2025

Weather ETL Pipeline - Apache Airflow

Production-ready, scalable ETL pipeline for real-time weather data collection, transformation, and storage using Apache Airflow, PostgreSQL, and Docker.

Apache AirflowPostgreSQLDockerDocker ComposePythonETLMLOpsREST APIJupyterData ValidationSQLGoogle Colab
View repository

Features

  • •Apache Airflow orchestration with TaskFlow API
  • •Real-time weather data extraction from Open-Meteo API
  • •PostgreSQL database with data quality views
  • •Docker Compose one-command deployment