Senior Data Engineer

Supercharge Your Career as a QA/Senior QA Engineer at Technoidentity! Are you ready to solve people challenges that fuel business growth? At Technoidentity, we’re a Data+AI product engineering company building cutting-edge solutions in the FinTech domain for over 15 years—and we’re expanding globally. It’s the perfect time to join our team of tech innovators and leave your mark! What’s in it for You? The Tax Compliance Data Engineering team at Technoidentity is at the heart of how our global tech platform client manages and scales its finance and compliance data infrastructure. This team sits within the Biztech Data Engineering organization and handles critical data related to Tax, Payments, and Legal Compliance regulations. Joining this team means working with cross-functional stakeholders, designing scalable solutions, and contributing to a world-class data engineering environment with a strong emphasis on quality, scalability, and robust engineering practices. What Will You Be Doing? Our team charter is centered on enabling our stakeholders to comply with Tax, Payments, and Legal regulations so that platform participants can continue operating in regulated geographies. The products built by this team ingest, process, validate, and deliver large datasets to government authorities, often in partnership with tax bodies, which means data accuracy and quality are non-negotiable. As you build and maintain key components of the critical compliance data ecosystem, you will have the opportunity to contribute to defining standards and best practices for Data Engineering at scale. Your work on solving complex business challenges will directly shape the tools, processes, and standards used by the broader data community. A Typical Day - Design, build, and maintain robust and efficient data pipelines that collect, process, and store data from various sources, including user interaction data, PII data, listing details, and external data feeds - Develop data models that enable efficient analysis; ensure data quality, consistency, and accuracy across all pipelines - Build high quality data assets for product use cases by partnering with cross-functional teams for compliance and reporting purposes - Build scalable data pipelines using SparkSQL and Scala, leveraging Airflow as the scheduler and executor framework - Collaborate with Data Scientists, Product Managers, and Software Engineers to define data requirements and deliver solutions that drive compliance and operational improvements - Contribute to the broader Data Engineering community to influence tooling and standards, improving culture and productivity - Improve code and data quality by leveraging and contributing to internal tools designed to automatically detect and mitigate issues Requirements What Makes You the Perfect Fit? 8+ years of relevant industry experience BE/B.Tech in Computer Science or a relevant technical discipline Hands-on experience in DSA coding (Data Structures and Algorithms) Extensive experience designing, building, and operating robust distributed data platforms (Spark, Kafka, Flink, HBase) with demonstrated ability to handle data at petabyte scale Strong understanding of Fintech, Ecommerce, or Booking Platform operating models Strong proficiency in Scala or Python, with deep expertise in data processing technologies and SQL query authoring Demonstrated ability to analyze large datasets to identify gaps and inconsistencies, surface insights, and drive effective data product solutions Expertise with ETL schedulers such as Apache Airflow, Luigi, Oozie, AWS Glue, or similar frameworks Solid understanding of data warehousing concepts and hands-on experience with relational databases (PostgreSQL, MySQL) and columnar databases (Redshift, BigQuery, HBase, ClickHouse) Excellent written and verbal communication skills