Note: The job is a remote job and is open to candidates in USA. Cotality is a company dedicated to transforming the property industry through data intelligence and technology. They are seeking a Director of Data Engineering & Platforms to lead data transformation initiatives, establish robust data architecture frameworks, and drive technical leadership in data engineering and AI enablement.
Responsibilities
- Lead the development and implementation of our data engineering strategy, architecture roadmap, and technical standards
- Oversee the design and evolution of our data ecosystem utilizing Data Vault methodologies and Data Mesh principles
- Establish governance and quality frameworks across Bronze (raw), Silver (transformed), and Gold (consumption-ready) data layers
- Partner with Product Management to align data platform capabilities with business objectives and market demands
- Drive the technical roadmap for data integration, transformation, and delivery systems, with explicit milestones for AI readiness
- Provide technical leadership and oversight for the data engineering team, ensuring best practices in data pipelines, transformations, and delivery
- Oversee the design and implementation of Snowflake data architecture including warehousing, marts, and access patterns optimized for both BI and AI workloads
- Direct the development of robust ETL/ELT processes using Matillion, Python-based pipeline frameworks, and other modern data integration tools
- Guide the implementation of data quality monitoring, lineage tracking, and metadata management
- Establish standards for data modeling, transformation logic, and performance optimization
- Partner with AI/ML and product engineering teams to ensure the data layer supports LLM-powered applications reliably and at scale
- Provide architectural direction for retrieval and grounding pipelines, including vector stores, embedding workflows, and hybrid search infrastructure
- Define standards for data preparation for AI, covering metadata enrichment, context optimization, and semantic indexing
- Build observability into AI data flows and monitor for drift and retrieval quality degradation
- Guide the team's evaluation and adoption of emerging AI-native data tools, including vector databases and LLM orchestration frameworks
- Establish governance frameworks for AI data use, including data lineage into models, PII controls upstream of LLM consumption, and output auditability
- Define the organization's standards for acceptable AI data quality thresholds and remediation workflows
- Partner with Security and Compliance to ensure AI data pipelines meet regulatory and privacy requirements
- Build, mentor, and lead a high-performing data engineering team with strong core data engineering fundamentals and a growing fluency in modern AI infrastructure
- Collaborate cross-functionally with Product Management, Data Analytics, Data Systems, Cloud Engineering, and product teams
- Foster a culture of innovation, continuous improvement, and technical excellence where AI is a tool the team uses daily, not a project they hand off
- Develop talent through coaching, training, and career development opportunities, with an emphasis on AI-era skills including Python, vector search, and agentic pipeline concepts
- Promote adaptive methodologies and DevOps practices within the data engineering discipline
- Oversee the technical implementation of Power BI reporting solutions and analytics platforms
- Ensure data pipelines efficiently support BI reporting needs and business intelligence requirements
- Partner with Data Analytics teams to optimize data structures for analytical workloads
- Guide the design of data models that enable self-service analytics and reporting
- Establish patterns for efficient and secure data access across the organization
- Evaluate emerging technologies and methodologies for potential integration into our data platform, with particular attention to AI/ML tooling and agentic workflow frameworks
- Lead proof-of-concepts and pilots for innovative data solutions, including AI-powered pipeline automation and LLM-grounded analytics
- Develop the technical foundation to support advanced analytics and machine learning initiatives
- Guide the evolution of our data architecture to support real-time and streaming use cases
- Stay current with industry trends and incorporate best practices into our data ecosystem
Skills
- Bachelor's degree from an accredited institution or equivalent professional experience with demonstrated capability
- 8+ years of progressive experience in data engineering, data architecture, or related technical roles
- 5+ years of leadership experience managing data engineering teams and initiatives
- Extensive experience with modern data platforms, particularly Snowflake and cloud-based data solutions
- Deep understanding of data modeling techniques including Data Vault, dimensional modeling, and Data Mesh concepts
- Hands-on experience with ETL/ELT tools like Matillion and data integration patterns
- Strong knowledge of SQL Server, Cosmos DB, and database technologies
- Experience with Power BI or similar BI platforms and understanding of reporting architectures
- Proven track record implementing data governance, quality, and metadata management solutions
- Experience partnering with product teams and translating business requirements into technical solutions
- Demonstrated interest in AI/ML data infrastructure, whether through independent projects, coursework, or applied experimentation
- Ability to engage credibly with engineers building LLM-powered systems and make sound architectural decisions without being the implementer
- Bachelor's or master's degree in computer science, Information Systems, or a related field
- Proficiency in Python and experience with Spark or other data processing frameworks
- Knowledge of CI/CD practices and DevOps for data pipelines
- Has independently explored or prototyped with vector databases, RAG pipelines, or LLM grounding concepts
- Familiarity with LLM orchestration frameworks such as LangChain or LlamaIndex
- Exposure to agentic workflow concepts and the data contracts they require
- Experience with real-time data integration and streaming architecture
- Background in implementing data security and privacy controls
- Understanding of API design and microservices architectures
- Experience in insurance, financial services, or real estate industries
Benefits
- Time off: Generous PTO and 11 paid holidays, plus well-being and volunteer time off.
- Family Support: Up to 16 weeks of fully paid parental leave and a baby stipend.
- Health: Multiple medical plan options with mental health and wellness support offerings.
- Retirement: 401(k) with company match and vesting after one year.
- Financial Perks: $400 annual well-being stipend and tuition assistance up to $5,250.
- Extras: Recognition Rewards, Referral bonuses, exclusive discounts and more!
Company Overview