We are seeking an experienced Spark Developer to design and implement the Gold layer of our Data Warehouse using the Medallion Architecture. The ideal candidate will have strong expertise in Apache Spark and a solid understanding of data engineering best practices.
Key Responsibilities:
Develop and optimise Spark-based data pipelines for the Gold layer.
Ensure data quality, performance, and scalability in the data warehouse architecture.
Collaborate with data analysts to deliver business-ready data sets.
Implement transformations and aggregations aligned with business requirements.
Technical Skills:
Mandatory experience with Airflow, Delta Lake , Java , Spark -> 3 years
Data Warehouse -> 5 years