Senior Data Engineer

Role Summary: Responsible for the design, automation, and delivery of large-scale financial datasets across cloud and on-prem platforms. Owns end-to-end data pipelines, data quality frameworks, and institutional client data onboarding, with a strong focus on automation, scalability, cost optimization, and reliable production delivery. Key Responsibilities: - Manage end-to-end onboarding and delivery of data products to institutional clients across cloud and on-prem platforms. - Work closely with product development and technology project team members to help propel data products across various asset-classes e.g. Equities, Fixed Income, Derivatives, FX, etc. - Design, develop, and optimize complex, highly scaled data pipelines to process and deliver streaming and historical data - improving computing efficiency, resolving performance bottlenecks, and building data-intensive applications to provide actionable insights to users and business partners. - Manage cloud infrastructure, secure data sharing and product listings on the AWS Data Exchange and Snowflake Marketplace, including access provisioning and cost optimization. - Use Python for development, along with SQL Server, Snowflake and AWS to consolidate data sources and Azure for deployment. - Develop and maintain Data Quality frameworks to ensure the accuracy, completeness, and integrity of data delivered to clients. - Lead initiatives to improve data operations activities to manage SLA: process automation, optimizing data delivery, and data infrastructure. - Build and deploy REST API-based data access solutions in Python. - Maintain familiarity with statistical and machine learning models and methods (distribution analysis, regression, classification, clustering, LLM, etc.) to support client queries and collaborate with data science and business teams. - Visualize and explore data sets to enable the idea generation of new, predictive features. - Investigate and evaluate new data frameworks and develop POCs for stock data analysis, cloud data delivery, and large-scale data processing projects. - Work with IT operations team for implementation including specification design and testing, applying modern software practices (unit testing, regression testing, version control). Position - Offshore (Support from India) Working Location - Remote - Support S3 Partners LCC from Ahmedabad (Gift City)