Note: The job is a remote job and is open to candidates in USA. Diverse Lynx is seeking a Data Engineer with strong expertise in Python-based data engineering and CI/CD automation using GitHub Actions. The candidate will play a critical role in building scalable data pipelines, automating deployments, and enabling high-quality data delivery across cloud platforms.
Responsibilities
- Design, build, and maintain scalable data pipelines using Python
- Develop ETL/ELT workflows for batch and near real-time data processing
- Work with structured and unstructured data across cloud/platform ecosystems
- Implement data transformation, validation, and quality frameworks
- Build and maintain CI/CD pipelines using GitHub Actions to automate: Code builds and validations, Unit/integration testing, Deployment of data pipelines and services
- Implement branching strategies, code reviews, and version control using Git
- Enable automated testing and release management for data workflows
- Integrate CI/CD pipelines with cloud platforms and data services
Skills
- Strong expertise in Python-based data engineering
- CI/CD automation using GitHub Actions
- Building scalable data pipelines
- Automating deployments
- Enabling high-quality data delivery across cloud platforms
- Designing, building, and maintaining scalable data pipelines using Python
- Developing ETL/ELT workflows for batch and near real-time data processing
- Working with structured and unstructured data across cloud/platform ecosystems
- Implementing data transformation, validation, and quality frameworks
- Building and maintaining CI/CD pipelines using GitHub Actions
- Automating code builds and validations
- Unit/integration testing
- Deployment of data pipelines and services
- Implementing branching strategies, code reviews, and version control using Git
- Enabling automated testing and release management for data workflows
- Integrating CI/CD pipelines with cloud platforms and data services
Company Overview
Diverse Lynx is a WBENC- and NMSDC-certified partner, helping organizations turn diversity goals into measurable impact through staffing and contingent workforce solutions. It was founded in 2002, and is headquartered in Princeton, New Jersey, US, with a workforce of 1001-5000 employees. Its website is http://www.diverselynx.com.Company H1B Sponsorship
Diverse Lynx has a track record of offering H1B sponsorships, with 1 in 2024, 1 in 2021. Please note that this does not guarantee sponsorship for this specific role.