Data Engineering Manager - AWS Databricks

We are looking for an experienced Data Engineering Manager with a strong foundation in Python, SQL, and Spark, and hands-on expertise in AWS, Databricks. In this role, you will build and maintain scalable data pipelines and architecture to support analytics, data science, and business intelligence initiatives. You’ll work closely with cross-functional teams to drive data reliability, quality, and performance.

Responsibilities:

• Design, develop, and optimize scalable data pipelines using Databricks in AWS such as Glue, S3, Lambda, EMR, Databricks notebooks, workflows and jobs.
• Building data lake in WS Databricks.
• Build and maintain robust ETL/ELT workflows using Python and SQL to handle structured and semi-structured data.
• Develop distributed data processing solutions using Apache Spark or PySpark.
• Partner with data scientists and analysts to provide high-quality, accessible, and well-structured data.
• Ensure data quality, governance, security, and compliance across pipelines and data stores.
• Monitor, troubleshoot, and improve the performance of data systems and pipelines.
• Participate in code reviews and help establish engineering best practices.
• Mentor junior data engineers and support their technical development.