[Remote] Data Engineer

Note: The job is a remote job and is open to candidates in USA. Prime Hiring Solutions (PHS) is a workforce solutions partner supporting U.S. clients with high-quality IT and engineering staffing. The Data Engineer will be responsible for designing, building, and maintaining enterprise knowledge ingestion pipelines to ensure seamless data integration from various sources.

Responsibilities

• Design, build, and maintain enterprise knowledge ingestion pipelines
• Develop and enhance web scraping solutions for authenticated content sources (AEM and enterprise portals)
• Implement content extraction, chunking, indexing, and vectorization workflows
• Manage AWS infrastructure using Terraform and Infrastructure as Code (IaC)
• Deploy and support ingestion services using GitLab CI/CD pipelines
• Configure and maintain OpenSearch Serverless, S3, DynamoDB, Lambda, ECS/Fargate, and Glue
• Implement monitoring, logging, alerting, dashboards, and operational runbooks
• Troubleshoot ingestion, infrastructure, and deployment issues across AWS services
• Optimize scalability, reliability, security, and cost of ingestion workloads
• Support RAG-based search and knowledge management solutions using Bedrock and vector search technologies

Skills

• Go (Golang)
• Python
• Node.js
• Lambda
• ECS/Fargate
• S3
• DynamoDB
• Glue
• OpenSearch Serverless
• IAM
• CloudWatch
• Bedrock
• RAG (Retrieval Augmented Generation)
• Vector Search
• Embeddings
• Document Chunking
• Content Indexing
• Knowledge Ingestion Pipelines
• Terraform
• GitLab CI/CD
• GitOps
• Infrastructure as Code (IaC)
• Secrets Management
• Monitoring & Observability
• Production Support
• Adobe Experience Manager (AEM)
• Web Scraping/Crawling
• Authenticated Content Sources
• Enterprise CMS Platforms
• Experience building enterprise search or knowledge management platforms
• Experience with Amazon Bedrock and LLM-powered applications
• Familiarity with permissions-aware content ingestion
• Strong debugging skills across distributed cloud systems
• Experience supporting production-grade data platforms

Company Overview

Prime Hiring connects global talents with dynamic roles from entry level to management across various industries. It was founded in undefined, and is headquartered in San Mateo, California, US, with a workforce of 11-50 employees. Its website is https://www.prime-hiring.com.

Company H1B Sponsorship

Prime Hiring Solutions has a track record of offering H1B sponsorships, with 1 in 2026. Please note that this does not guarantee sponsorship for this specific role.