Azure Data Engineer

Role Overview We are looking for experienced Azure Databricks Data Engineers with strong hands-on expertise in Apache Spark/PySpark, Python, SQL, and Azure Data Factory (ADF) . The candidate will be responsible for designing, developing, and optimizing scalable data engineering solutions on the Azure cloud platform. Mandatory Skills - Strong hands-on experience with Azure Databricks - Strong knowledge of Apache Spark and/or PySpark - Proficiency in Python - Strong SQL development and query optimization skills - Hands-on experience with Azure Data Factory (ADF) - Experience developing and maintaining scalable ETL/ELT data pipelines - Good understanding of Azure data engineering concepts and cloud-based data platforms Key Responsibilities - Design, develop, and maintain data pipelines using Azure Databricks and ADF - Develop efficient data processing solutions using PySpark/Spark and Python - Write complex SQL queries for data transformation, validation, and analysis - Build scalable ETL/ELT workflows for large datasets - Optimize Spark jobs, Databricks notebooks, and data pipelines for performance - Integrate data from multiple sources into Azure-based data platforms - Implement data quality, validation, error handling, and monitoring mechanisms - Troubleshoot production issues and provide timely resolutions - Collaborate with data architects, analysts, developers, and business stakeholders - Follow best practices for code quality, security, performance, and maintainability Experience Requirements For 5+ Years - 5+ years of overall experience in data engineering - Strong hands-on experience in Azure Databricks, PySpark/Spark, Python, SQL, and ADF - Experience working on enterprise-scale data pipelines For 9+ Years - 9+ years of overall experience in data engineering - Strong expertise in Azure Databricks and modern Azure data engineering - Proven experience designing and optimizing large-scale data pipelines - Ability to lead technical discussions and mentor junior engineers Preferred Skills - Azure Data Lake Storage (ADLS) - Delta Lake - Databricks Workflows - Azure DevOps / CI-CD - Git - Data warehousing concepts - Experience with Agile/Scrum methodologies Skills:- Azure Databricks, Azure data factory, Python, PySpark, Apache Spark and SQL