Staff Engineer (Cloudera CDP & Hadoop Administration)

Job Description Requirements - Experience : 5.5+ years - Hands-on Tier-2 production experience in Cloudera CDP and Hadoop administration . - Strong experience installing, configuring, maintaining, and troubleshooting Cloudera CDP environments. - Experience with Cloudera CDP 7.1.9 or later and Cloudera Data Services, including versions 1.5.4 and 1.5.5. - Strong understanding of Hadoop ecosystem components, particularly Spark, Hive, and YARN . - Proven experience troubleshooting Spark, Hive, and YARN jobs in production environments. - Experience optimizing Cloudera cluster performance, resource utilization, and workload execution. - Strong understanding of CDP security configuration and implementation of security and regulatory requirements. - Hands-on experience with automation using Ansible, Shell scripting, Terraform, Groovy, and Python . - Experience building and managing automated CI/CD pipelines and workflows using GitHub Actions, Jenkins, ArgoCD , or equivalent tools. - Good understanding of GitOps practices and source control using Git. - Experience with containerization technologies such as Podman, Docker, Kubernetes, and OpenShift/OCP . - Knowledge of Infrastructure as Code (IaC) concepts and automated infrastructure provisioning. - Experience implementing observability and monitoring for data platforms and production environments. - Experience automating security, compliance, regulatory checks, and patching activities. - Strong Linux administration and basic networking knowledge. - Experience working in Agile engineering and DevOps-oriented environments. - Strong troubleshooting, analytical, and problem-solving skills with the ability to support critical production data platforms. - Ability to collaborate effectively with data engineers, infrastructure teams, security teams, and other technical stakeholders. Responsibilities - Administer, install, configure, maintain, and support Cloudera CDP and Hadoop environments in Tier-2 production environments. - Perform installation, configuration, upgrades, patching, and ongoing maintenance of the latest supported Cloudera CDP versions, including CDP 7.1.9 or later. - Configure and implement security requirements across Cloudera CDP environments, including automated security and regulatory compliance checks. - Configure and maintain Cloudera Data Services, including versions 1.5.4, 1.5.5, and later supported releases. - Troubleshoot and resolve production issues related to Spark, Hive, and YARN jobs. - Monitor and optimize cluster health, resource utilization, capacity, and overall CDP performance. - Work closely with data engineering teams to troubleshoot and optimize Spark job performance and resource utilization. - Implement automation for cluster administration, operational processes, security checks, patching, and data/analytics workloads. - Develop automation scripts and infrastructure workflows using Shell, Ansible, Terraform, Groovy, and Python. - Design and maintain CI/CD pipelines and automated workflows for data projects using tools such as GitHub Actions, Jenkins, and ArgoCD. - Implement GitOps-based deployment and operational practices for data and analytics platforms where applicable. - Support containerized Cloudera Data Services using technologies such as Podman, Docker, Kubernetes, and OpenShift Container Platform (OCP). - Automate infrastructure provisioning and configuration using Infrastructure as Code (IaC) practices. - Implement and maintain observability solutions for Cloudera clusters, data services, jobs, and infrastructure components. - Perform automated patching and support lifecycle management activities across the platform. - Collaborate with engineering, data, infrastructure, security, and operations teams to resolve production issues and improve platform reliability. - Maintain source code, automation scripts, configurations, and deployment artifacts using Git-based source control. - Follow Agile engineering practices and contribute to continuous improvement, operational automation, and platform standardization.