Data Engineer

ROLE SUMMARY

We are looking for a Data Engineer focused on the hands-on execution and technical delivery of data solutions. Reporting to the solution architect, he will build, optimize, and maintain data pipelines and Lakehouse structures within the Microsoft Fabric ecosystem, converting architectural designs into high-performance, scalable and efficient data delivery

DESCRIPTION OF TASKS

• Delta Lake implementation: Build and optimize physical storage layers (Bronze/Silver/Gold) using Delta Lake formats, ensuring proper partitioning and Z-Ordering.

• Pipeline execution: Construct robust ETL/ELT workflows using Fabric Data Factory, Spark, and custom code to move data from diverse sources into OneLake.

• Custom application development: Write and maintain high-performance data processing modules or custom connectors using Java and/or Python.

• Cross-platform integration: Implement data bridges between Microsoft Fabric and external environments (e.g., AWS, GCP, or on-premises systems) using Shortcuts and APIs.

• Code-based transformations: Develop sophisticated logic within Spark Notebooks to handle complex data cleansing and business logic application.

• System testing: Perform rigorous unit testing, data profiling, and performance benchmarking to ensure pipeline stability and data integrity.