Confluent Integration Engineer

This role is for one of the Weekday's clients Salary range: Rs 600000 - Rs 1000000 (ie INR 6 - 10 LPA) Min Experience: 3+ years Location: Pune JobType: full-time We are looking for a skilled Confluent Integration Engineer with 3–6 years of experience in designing, developing, implementing, and supporting real-time data integration solutions using Apache Kafka and Confluent Platform . The ideal candidate will have strong hands-on expertise in Kafka-based architectures, Confluent connectors, data pipelines, and event-driven integration patterns. You will work closely with engineering, application, data, and infrastructure teams to build reliable, scalable, and high-performance streaming solutions. The role requires strong troubleshooting skills and the ability to manage Kafka and Confluent integrations across development, testing, and production environments. Requirements Key Responsibilities - Design, develop, configure, and maintain real-time data integration pipelines using Apache Kafka and Confluent Platform . - Implement and manage Kafka Connect frameworks and Confluent connectors for integrating databases, applications, APIs, cloud platforms, and enterprise systems. - Configure and maintain Source and Sink Connectors , including connector properties, transforms, converters, tasks, and worker configurations. - Develop scalable event-driven integration solutions using Kafka topics, partitions, consumer groups, producers, and consumers. - Monitor Kafka clusters, Connect workers, connectors, and data pipelines to ensure availability, reliability, and performance. - Troubleshoot connector failures, message delivery issues, consumer lag, serialization problems, schema-related issues, and pipeline disruptions. - Work with Confluent Schema Registry to manage Avro, JSON Schema, or Protobuf schemas and maintain data compatibility. - Optimize Kafka configurations including partitions, replication, retention, consumer settings, and connector task allocation. - Implement data transformation and routing requirements using Kafka Connect Single Message Transforms (SMTs) where appropriate. - Collaborate with application and data teams to understand integration requirements and translate them into robust Kafka-based solutions. - Support deployment, configuration management, upgrades, and production releases for Kafka and Confluent environments. - Establish monitoring, logging, alerting, and operational processes for Kafka and connector infrastructure. - Create technical documentation covering architecture, configurations, integration workflows, troubleshooting procedures, and operational standards. Must-Have Skills - 3–6 years of hands-on experience in Kafka and enterprise data integration. - Strong expertise in Apache Kafka concepts, architecture, and administration. - Hands-on experience with Confluent Platform and Confluent Kafka . - Strong experience configuring and managing Kafka Connect and Confluent Connectors . - Experience working with both Source and Sink Connectors . - Good understanding of Kafka producers, consumers, topics, partitions, replication, offsets, and consumer groups. - Experience with Confluent Schema Registry and serialization formats such as Avro, JSON Schema, or Protobuf. - Strong troubleshooting and performance-tuning capabilities for Kafka and connector-based integrations. - Good understanding of REST APIs and integration patterns. - Proficiency in scripting or programming using Java, Python, or similar languages . Good-to-Have Skills - Experience with connectors for databases such as Oracle, PostgreSQL, MySQL, or SQL Server . - Exposure to cloud platforms such as AWS, Azure, or GCP . - Experience with Kafka security, including SSL/TLS, SASL, ACLs, and authentication mechanisms . - Knowledge of Kubernetes and containerized Confluent deployments. - Familiarity with monitoring tools such as Prometheus, Grafana, Splunk, or ELK. - Understanding of CI/CD pipelines and DevOps practices. - Experience with Kafka Streams or ksqlDB is an advantage. Qualifications Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related discipline, along with strong analytical, communication, and problem-solving skills.