AWS Glue Data Pipeline Tester

The candidate should be able to define and validate data quality rules covering:

• Completeness
• Accuracy
• Consistency
• Validity
• Uniqueness
• Timeliness
• Referential integrity
• Conformity to business rules
• Schema and metadata consistency

Testing Areas

The role will include the following types of testing:

• Functional testing
• End-to-end integration testing
• Data validation testing
• Source-to-target reconciliation
• Parallel Testing with legacy interfaces testing.
• Regression testing
• System integration testing
• User acceptance testing support
• Negative testing
• Exception and error-path testing
• Performance and volume testing
• Scalability testing
• Restart and recovery testing
• Failover and resiliency testing
• Security and access testing
• Production smoke testing
• Operational and monitoring validation

Requirements

• AWS Glue Jobs
• AWS Glue Workflows
• Amazon S3
• Amazon CloudWatch
• AWS IAM
• Amazon EventBridge
• AWS Lambda
• Amazon SNS
• Amazon SQS
• AWS Secrets Manager
• AWS Step Functions
• AWS KMS
• Ability to review AWS Console configurations, job parameters, logs, metrics, and execution history.
• Data Pipeline and File Processing Interface Testing
• Strong understanding of Inbound and Outbound Data File Processing / Data Pipeline.
• Experience testing inbound/outbound file transfers across S3, SFTP/FTPS, AWS Transfer Family, APIs, and external partner endpoints, including authentication, connectivity, retry, timeout, and delivery validation.