Software Engineer II - Level 8

Summary The Backend Software Engineer will join the Company&rsquos Data Engineering Platform team and play a key role in building and evolving large-scale backend data systems, real-time and batch processing pipelines, and AI-enabled services. This position focuses on backend engineering using Scala, Python, and Java, distributed data platforms including Cloudera and Spark, and cloud-based architectures. The role also contributes to AI-powered automation initiatives that support analytics, decision-making, and operational efficiency across the organization. The ideal candidate will have experience working with large-scale datasets, modern data platforms, and cloud technologies while collaborating with cross-functional teams to deliver reliable and scalable backend solutions. Responsibilities - Design, develop, and maintain backend services and data pipelines using Scala, Python, and Java. - Build, optimize, and support batch and streaming workloads on Cloudera Data Platform (CDP) using Spark. - Configure, monitor, troubleshoot, and improve platform performance using Cloudera Manager. - Design and implement high-quality DataMart&rsquos and curated datasets with a focus on data integrity, reliability, and performance. - Develop and integrate AI-powered agents that support intelligent automation, operational efficiency, and scalable workflows. - Contribute to anomaly detection, operational intelligence, and workflow automation initiatives. - Support cloud-based data and compute workloads within AWS environments. - Utilize Databricks for large-scale data processing and advanced analytics workloads. - Contribute to cloud-native and hybrid architectures that integrate on-premises and cloud-based platforms. - Enable downstream consumers, including analytics, reporting, and visualization tools, through reliable and scalable backend data interfaces. - Collaborate with analytics, business, and technical teams to understand requirements and deliver backend solutions that support evolving business needs. - Create and maintain technical documentation, including architecture diagrams, design documents, and data flow specifications. - Participate in Agile and Scrum ceremonies, design reviews, and cross-functional planning activities. - Support knowledge sharing and mentor team members in backend engineering, big data technologies, and AI-related concepts. - Continuously evaluate opportunities to improve platform scalability, automation, stability, and efficiency. Education Requirements - Bachelor&rsquos degree in Computer Science, Engineering, Information Systems, or a related field, or equivalent professional experience. Experience Requirements - Strong hands-on experience developing backend or data-intensive systems using Scala, Python, and/or Java. - Experience working with Cloudera Data Platform (CDP) and Spark. - Experience supporting large-scale distributed data processing environments. - Experience working in Agile or Scrum-based development teams. Required Skills - Proficiency in Scala, Python, and Java for backend development. - Experience with Cloudera Data Platform (CDP). - Strong knowledge of Apache Spark. - Familiarity with Cloudera Manager for cluster administration, monitoring, and troubleshooting. - Strong understanding of distributed systems and large-scale data processing. - Strong data modeling and data engineering skills. - Experience designing, developing, and supporting data pipelines. - Excellent analytical and problem-solving abilities. - Ability to work independently in complex technical environments. - Strong verbal and written communication skills. - Experience creating technical documentation and design specifications. - Ability to collaborate effectively with cross-functional stakeholders. Preferred Skills - Experience building, integrating, or supporting AI-driven agents and intelligent automation solutions. - Experience with Databricks for data engineering or machine learning workloads. - Experience working within AWS environments, including services such as S3, EC2, EMR, Glue, Lambda, IAM, or similar technologies. - Knowledge of streaming and big data technologies, including Kafka. - Knowledge of Hadoop ecosystem technologies. - Experience with Hive and Impala. - Exposure to model monitoring or AI platform enablement activities. - Experience with ETL tools such as Informatica. - Experience working within large enterprise environments. - Familiarity with modern cloud and hybrid architecture patterns. #AditiConsulting # 26-05061 Skills: Data Modeling, Scala, Apache Spark, Distributed Systems, Cloudera Manager, data engineering , Java, Aws, Databricks, Python Experience: 0.00-0.00 Years