Note: The job is a remote job and is open to candidates in USA. Emory University is a leading research university that fosters excellence and attracts world-class talent to innovate today and prepare leaders for the future. The Data Engineer II is responsible for developing and evolving the Unified Data Platform, working closely with researchers and cross-functional teams to deliver scalable data solutions that support research and analytics.
Responsibilities
- Collaborate as a core member of cross-functional teams including Business Analysts, Project Managers, Data Analysts, and Architects to deliver high-quality solutions within scope and timeline
- Work directly with researchers and stakeholders to gather, analyze, and translate requirements into technical solutions
- Design, develop, and maintain scalable data pipelines, ETL/ELT processes, and data integration workflows across multiple systems
- Build and optimize data solutions that integrate data from disparate sources, ensuring data quality, integrity, and consistency
- Evaluate emerging technologies and develop proof-of-concepts to support innovation and continuous improvement
- Apply biomedical informatics standards, methodologies, and principles to research data solutions
- Ensure adherence to HIPAA and institutional data governance policies and standards
- Develop and maintain metadata, data standards, and data governance processes for complex datasets
- Create and maintain clear technical documentation, including data pipelines, workflows, and system designs
- Communicate technical concepts effectively to both technical and non-technical stakeholders
- Partner with data and system architects and demonstrate an understanding of data modeling concepts and best practices
- Manage workload effectively and provide timely updates on task progress and deliverables
- Works as a positive team member of a project that may consist of Business Analysts, Project Managers, Information Architects, Data Analysts, and/or Database Administrators to deliver quality applications and components within scope, on time, and within budget
- Manages workload effectively and report status of tasks in a timely manner
- Works directly with researchers to document, analyze, and translate their needs into technical designs and informatics solutions
- Participates in the evaluation of emerging technologies and develops proof-of-concepts
- Contributes to technical teams
- Follows standard operational procedures and HIPAA regulations
- Develops strategies for managing complex data sets through maintaining data standards and metadata
- Applies biomedical informatics technical standards, methodologies, and principles to research-specific program needs, objectives, and outcomes
- Develops complex reports, data pipelines, and ETL processes from disparate systems and ensures their accuracy
- Gathers user requirements and creates technical documentation
- Performs other related duties as required
Skills
- A bachelor's degree in a related field and three years of related experience, OR an equivalent combination of education, training, and experience
- Strong experience designing and maintaining data pipelines and ETL/ELT processes
- Proficiency in Python, SQL, and Apache Spark (PySpark)
- Experience with modern data platforms such as Azure Fabric (preferred), Azure Synapse Serverless, or Databricks
- Ability to work with large, complex, and distributed datasets
- Solid understanding of data modeling concepts and best practices
- Familiarity with data governance, metadata management, and data quality frameworks
- Knowledge of HIPAA and healthcare data compliance standards
- Experience with CI/CD practices and proficiency with version control systems like Git
- Familiarity with data pipeline orchestration and monitoring tools Like Airflow, Azure Data Factory or similar frameworks
- Understanding of biomedical informatics methodologies is a plus
- Strong analytical, problem-solving, and critical-thinking abilities
- Excellent communication and collaboration skills across diverse teams
- Ability to adapt quickly to new technologies and evolving data environments
- Exposure to FHIR, OMOP, or similar healthcare data models (nice-to-have)
Company Overview
Company H1B Sponsorship