Data Engineer - AzureExperienced Associate/Senior Associate/Manager

Role Summary

We are seeking skilled Data Engineers with prior experience on Azure data engineering services and SQL server to join our team. As a Data Engineer, you will be responsible for designing and implementing robust data infrastructure, building scalable data pipelines, and ensuring efficient data integration and storage.

Responsibilities

• Design, develop, and maintain scalable data pipelines and use Azure Data Factory and Azure Stream Analytics
• Collaborate with data scientists and analysts to understand data requirements and implement solutions that support analytics and machine learning initiatives
• Optimize data storage and retrieval mechanisms to ensure performance, reliability, and cost-effectiveness
• Implement data governance and security best practices to ensure compliance and data integrity
• Troubleshoot and debug data pipeline issues, providing timely resolution and proactive monitoring
• Stay abreast of emerging technologies and industry trends, recommending innovative solutions to enhance data engineering capabilities

Qualifications

• Proven experience as a Data Engineer or in a similar role
• Experience in designing and hands-on development in cloud-based (AWS/Azure) analytics solutions
• Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, Azure App Service, Azure Databricks, Azure IoT, Azure HDInsight + Spark, Azure Stream Analytics
• Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential
• Experience in SQL server and procedures
• Thorough understanding of Azure Infrastructure offerings
• Strong experience in common data warehouse modelling principles including Kimball, Inmon
• Experience in additional Modern Database terminologies
• Working knowledge of Python or Java or Scala is desirable
• Strong knowledge of data modelling, ETL processes, and database technologies
• Experience with big data processing frameworks (Hadoop, Spark) and data pipeline orchestration tools (Airflow)
• Solid understanding of data governance, data security, and data quality best practices
• Strong analytical and problem-solving skills, with attention to detail