Case Study
Smart Retail Lakehouse (Azure Databricks)
End-to-end data platform on Azure Databricks, with Medallion architecture and Kimball modeling.
Overview
Data engineering project that simulates a retailer, generating synthetic data and implementing a complete Lakehouse (Bronze → Silver → Gold) with Delta Lake, Unity Catalog, Workflows and MLflow.
Challenge
Demonstrate, reproducibly, a modern analytical platform covering ingestion, data quality, dimensional modeling, governance and experiment tracking — without relying on sensitive corporate data.
Solution
A PySpark pipeline was built following the Medallion pattern, with data-quality validation and record rejection, a Gold star-schema layer (Customer, Product, Store, Date dimensions and a Sales fact), Unity Catalog governance, Databricks Workflows orchestration and MLflow for experiments.
Impact
- Complete, reproducible Lakehouse architecture with synthetic data
- Kimball dimensional modeling ready for BI consumption
- Governance, data quality and ML tracking with MLflow