Big Data Engineer

Sr Level Python developer with 8 years of software development experience Build

and maintain ETL pipelines using Python and PySpark on AWS Glue and related

platforms. - Orchestrate workflows using AWS Step Functions and Lambda. -

Implement messaging and event-driven integrations using SNS and SQS. - Design

and optimize storage and querying solutions in Amazon Redshift, RDS, Oracle and

S3-based architectures. - Write efficient SQL for transformations, validation,

and reporting. - Integrate data from APIs and process structured and

semi-structured JSON data. - Implement data quality checks, monitoring, and

operational support processes. - Participate in CI/CD and version control

practices for deployment and release management. - Collaborate with

cross-functional teams to translate business requirements into technical

solutions. - 4-8 years of software development experience across the appropriate

platform. - Strong hands-on experience with Python, PySpark, API’s and SQL. -

Experience with ETL/data pipeline development and Orchestration using Step

functions / AirFlow . - Working knowledge of AWS services including Glue,

Lambda, Step Functions, Redshift, S3, SNS, and SQS. - Experience with Athena,

EMR, Kinesis, DynamoDB, or RDS. - Good Knowledge on CloudWatch, logging, and

production support. - Understanding of data warehousing, data lakes, Lake House

and query optimization. - Experience with GitLab/Terraform or similar and CI/CD

workflows. - Good understanding of using AI tools like Github Copilot or similar

for code productivity - Exposure to enterprise data lake or cloud migration

initiatives. - Have an eye to solving complex problems, great communication with

stakeholders - Have a good understanding of performance engineering of code

pipelines and near real time systems - Good understanding on Agents and MCP"