DE&A - Core - Data Quality Management - Data Quality Management (Other)

Description We are seeking a hands on QA Lead to drive quality assurance for a scalable, enterprise-wide data platform for an insurance client. The role involves validating batch and real-time data pipelines , ensuring data accuracy across Raw, Silver, and Gold layers , and supporting report rationalization and self-service analytics (Power BI) . The QA Lead will define and implement end-to-end data testing strategies , covering ingestion (AWS Glue, Kinesis), transformation (DBT), and consumption layers, while ensuring data quality, integrity, and performance optimization . Key Responsibilities 1. QA Strategy & Leadership - Define and implement end-to-end QA strategy for the enterprise data platform - Establish test frameworks, standards, and governance for data validation - Lead QA planning, estimation, and execution across multiple data streams 2. Data Validation & Testing - Validate data across Raw, Silver, and Gold layers ensuring accuracy, completeness, and consistency - Perform source-to-target reconciliation for batch and real-time pipelines - Design and execute: - Data quality checks - Transformation validation (DBT models) - Aggregation and KPI validation 3. Batch & Real-Time Pipeline Testing - Test batch ingestion pipelines using AWS Glue - Validate real-time streaming data pipelines using Amazon Kinesis - Ensure data latency, sequencing, and event consistency in streaming pipelines 4. Reporting & Rationalization QA - Validate datasets powering Power BI self-service reports - Support report rationalization initiatives by ensuring consistency of KPIs and eliminating redundant data sources - Perform report/data reconciliation testing across legacy vs new platform 5. Automation & Tools - Develop and implement automated data testing frameworks - Leverage SQL, Python, and testing tools (e.g., Great Expectations, DBT tests, custom frameworks) - Enable continuous testing integration within CI/CD pipelines 6. Performance & Optimization Testing - Validate performance of: - Data pipelines - Queries in Snowflake - Identify bottlenecks and work with engineering teams to optimize pipelines and queries - Ensure scalability for large data volumes and concurrent workloads 7. Data Quality & Governance - Define and enforce data quality rules, thresholds, and monitoring - Implement data anomaly detection and alerting mechanisms - Ensure compliance with audit, reconciliation, and governance standards Required Skills & Experience Core Technical Skills - Strong experience in data testing / ETL testing / data QA - Hands-on expertise with: - Snowflake (data validation, SQL testing) - DBT (testing, model validation) - AWS Glue (batch pipeline validation) - Amazon Kinesis (real-time pipeline testing) - Advanced proficiency in SQL for data validation and reconciliation - Programming skills in Python (preferred) Testing Expertise - Experience in: - Data reconciliation (source vs target) - Data quality frameworks and validation techniques - Automated data testing tools - Understanding of medallion architecture (Raw, Silver, Gold layers) Analytics & Reporting - Experience validating Power BI reports and datasets - Strong understanding of business KPIs and reporting consistency Domain Expertise (Preferred) - Experience in Insurance domain (Policy, Claims, Billing data) - Familiarity with regulatory reporting, audit, and reconciliation requirements Experience - 8–12 years in QA / Data Testing / ETL Testing - 3+ years in QA leadership or lead role - Experience working on enterprise-scale data platforms Responsibilities QA Lead Qualifications QA Lead