Lead / Manager - Data Engineering

Job Description We are looking for an experienced Manager – Data Engineering with 8+ years of experience in building and managing data engineering solutions in an on-premise environment . The ideal candidate should have strong hands-on expertise in Python, Apache Spark, SQL, and Apache Airflow , along with proven experience in leading data engineering teams and delivering scalable data pipelines. Key Responsibilities - Lead and manage a team of Data Engineers and provide technical guidance and mentorship. - Design, develop, and optimize scalable data pipelines in an on-premise environment. - Build robust ETL/ELT workflows using Python, Spark, SQL, and Airflow . - Design and manage complex Apache Airflow DAGs for data pipeline orchestration. - Develop and optimize Spark-based data processing solutions for large datasets. - Write complex and optimized SQL queries for data extraction, transformation, and analysis. - Troubleshoot pipeline failures, performance issues, and data quality challenges. - Work closely with architects, business stakeholders, and cross-functional teams to understand requirements and deliver solutions. - Conduct code reviews and ensure adherence to engineering and development best practices. - Drive technical design, estimation, planning, and end-to-end project delivery. - Monitor team performance, project timelines, risks, and dependencies. - Mentor engineers and contribute to building a strong data engineering practice.