Medallion Architecture for Data Engineering projects

Upstox trading analysis notebook with Medallion architecture — insights into portfolio performance.

Declarative Implementation of Medallion Architecture - using Opinionated Open-source libraries at different Layers

End-to-end ELT pipeline built on Snowflake using Medallion Architecture (Bronze → Silver → Gold) with Streams & Tasks, RBAC, and Time Travel

Data Warehouse in MySQL using Medallion Architecture

This repository contains a production-grade Medallion Architecture data pipeline built with Databricks Spark Declarative Pipelines (SDP), ingesting data from AWS S3 and transforming it through Bronze, Silver, and Gold layers. The pipeline processes customer, product, and sales data with built-in data quality checks

An end-to-end Netflix data engineering pipeline built on Microsoft Azure. This project ingests raw Netflix data, applies PySpark transformations , enforces data quality with Delta Live Tables, and orchestrates workflows via Azure Data Factory and Databricks.

An end‑to‑end data engineering pipeline for NYC Green Taxi trip records, built on Microsoft Azure. This project ingests Jan–Dec 2024 Parquet files from the NYC Taxi API into a Bronze Delta Lake layer, cleans and enriches the data in a Silver layer with PySpark on Azure Databricks, then saves the transformed data to the Gold layer in delta format

Automated job hunting pipeline — scrapes vacancies, filters and scores them with AI, and delivers the best matches via Telegram.

A modern data warehouse using the batch processing. And ETL implementation on CRM and ERP data.