Senior Data Engineer - (Flink/Kafka/PySpark)

We are looking for an experienced Data Engineer with strong expertise in real-time data streaming and distributed data processing technologies. The ideal candidate should have hands-on experience in building scalable, event-driven data platforms using Apache Flink, Kafka, Java/Scala, and PySpark. The candidate will be responsible for designing, developing, and maintaining high-performance data engineering solutions within large-scale enterprise environments. Requirements Key Responsibilities: - Design, develop, and maintain real-time streaming data pipelines and event-driven architectures. - Build scalable and fault-tolerant data engineering solutions using Apache Flink, Kafka, and Java/Scala. - Develop and optimize distributed data processing applications for enterprise-grade systems. - Implement best practices for performance tuning, resiliency, security, and compliance requirements. - Collaborate with cross-functional teams to design technical and application architectures. - Ensure production-grade monitoring, troubleshooting, and operational excellence of distributed services. - Contribute to CI/CD implementation and DevOps best practices. - Work closely with engineering teams to drive technical improvements and architectural decisions. Mandatory Technical Skills: - Minimum 4+ years of development and design experience in: - Apache Flink - Java or Scala - Apache Kafka - PySpark - Real-time data streaming technologies - Event-driven architectures Hands-on experience with: - Apache Flink (Beam or Spark Streaming experience is also valuable) - Kafka ecosystem and streaming platforms - JVM tuning and performance optimization - Distributed systems design and implementation - Docker and Kubernetes containerization - Linux OS administration and Shell scripting - SQL and NoSQL databases - CI/CD tools such as GitHub and Jenkins - Design patterns and their implementation - Production monitoring and troubleshooting of distributed services Nice to Have Skills: - Redis or other caching technologies. - Experience with Spark Streaming or Apache Beam. - Experience working with banking or fintech platforms. Data Engineering & Security Requirements: - Strong understanding of data security principles and controls. - Experience implementing secure data transfer mechanisms including: - CRON jobs - ETL processes - JDBC and ODBC scripts Knowledge of encryption, anonymization, data integrity, and policy controls in large-scale infrastructures.Understanding of data sensitivity requirements, including: - Protection of PII data - Secure logging practices - Secure in-memory data handling Ability to identify security design gaps and recommend enhancements.Experience implementing wrapper solutions for legacy or third-party components to ensure compliance requirements are met. Infrastructure & Networking Knowledge: - Working knowledge of: - DNS - Proxy servers - ACLs - Networking policies - Troubleshooting network-related issues Functional Requirements: - Experience working in Agile environments. - Ability to design and implement scalable technical architectures. - Conduct technology research and benchmarking activities. - Ability to influence engineering teams on technical best practices. - Experience working in enterprise-scale environments. - Banking, Financial Services, or FinTech domain experience is preferred. Soft Skills: - Excellent communication and interpersonal skills. - Strong problem-solving and analytical abilities. - Self-driven and capable of working independently. - Ability to collaborate effectively with cross-functional teams and stakeholders. - Strong presentation and listening skills. - Positive attitude and ability to foster a collaborative team environment. - Passionate about engineering excellence and continuous improvement.