Big Data Engineer

We are seeking a skilled Big Data Engineer to join our team in India. The ideal candidate will have extensive experience in designing and implementing robust big data solutions that support our data-driven decision-making processes. Job description - Design, build, and maintain scalable ETL/ELT pipelines using AWS Glue, Lambda, Step Functions, and Apache Spark . - Develop and optimize data lake and data warehouse architectures using AWS S3, Redshift, and Athena . - Implement data ingestion, transformation, and storage solutions using AWS Glue, Kinesis, and Kafka . - Work with structured and unstructured data to develop efficient data pipelines . - Optimize query performance in SQL-based and NoSQL databases (Redshift, DynamoDB, Aurora, PostgreSQL, etc.). - Ensure data security, compliance, and governance using IAM roles, KMS encryption, and AWS Lake Formation . - Automate data workflows using Terraform, CloudFormation, or CDK . - Monitor and troubleshoot data pipelines, ensuring high availability and performance. - Collaborate with Data Scientists, Analysts, and Engineers to provide robust data solutions. Required Skills: - Strong experience with AWS Data Services (Glue, Redshift, S3, Lambda, Kinesis, Athena). - Proficiency in Python, SQL, and Scala - Experience with ETL/ELT pipeline design and data warehousing concepts. - Strong knowledge of relational and NoSQL databases (PostgreSQL, MySQL, DynamoDB). - Familiarity with DevOps practices and CI/CD tools (Jenkins, GitHub Actions). - Experience with Kafka or Kinesis for real-time data streaming. - Knowledge of Terraform, CloudFormation, or CDK for infrastructure automation. - Hands-on experience with data security and governance . - Strong analytical and problem-solving skills Skills: Data Modeling, Spark, Scala, Kafka, Etl Tools, Sql, Aws, Python, Hadoop Experience: 6.00-11.00 Years