Back to projects
2025

Weather ETL Pipeline - Apache Airflow

Production-ready, scalable ETL pipeline for real-time weather data collection, transformation, and storage using Apache Airflow, PostgreSQL, and Docker.

Apache AirflowPostgreSQLDockerDocker ComposePythonETLMLOpsREST APIJupyterData ValidationSQLGoogle Colab
View repository

Features

  • -Apache Airflow orchestration with TaskFlow API
  • -Real-time weather data extraction from Open-Meteo API
  • -PostgreSQL database with data quality views
  • -Docker Compose one-command deployment