Senior Data Engineer [T500-25117]

About Comply: Comply is a global leader in regulatory compliance software for financial services, offering a scalable SaaS platform backed by expert consulting. Serving thousands of firms worldwide, Comply empowers broker-dealers, investment banks, RIAs, and more to manage compliance effectively. At Comply India, engineers build impactful, client-focused solutions using AI and emerging technologies, with strong ownership and growth opportunities.

Senior Data Engineer (Data & Analytics) Data and Analytics at COMPLY: COMPLY is the world’s leading aggregator of financial and regulatory data to support compliance. Our mission is to help financial institutions meet their regulatory obligations with confidence, clarity, and speed. We ingest, process, and enrich vast volumes of complex, high-variance data from hundreds of brokers, data providers and other sources of information across the US and beyond.

The scale, diversity, and importance of this data creates unique technical challenges and opportunities for innovative engineers. The COMPLY Data Platform is a strategic initiative at the heart of this mission. It underpins everything from our broker feed ingestion and security master to advanced analytics, AI-driven insights, and future regulatory technologies.

We’re building something foundational, and we’re doing it with a modern, cloud-native architecture that’s designed to scale. We’re looking for an experienced Senior Data Engineer to help us shape and grow this platform. The Role: As a Senior Data Engineer, you will design, build, and optimize the data pipelines and models that power Comply’s data platform.

You’ll focus on ingestion, transformation, modelling, quality, and data reliability — ensuring that downstream consumers (analytics, services, UI, AI) can depend on trusted, well-structured data. You’ll work closely with backend engineers, product teams, data operations, and architects to design data flows that are scalable, observable, and resilient. This role is deeply hands-on and focused on data engineering excellence.

Key Responsibilities: Design and implement robust batch and streaming data pipelines. Develop and maintain ingestion frameworks for broker feeds and third-party data sources. Build and optimize data transformation workflows (ETL/ELT).

Design scalable data models (warehouse/lakehouse patterns). Implement data quality validation, reconciliation, and monitoring. Contribute to metadata, lineage, and governance frameworks.

Optimize performance and cost of data processing workflows. Improve reliability through observability, testing, and automated monitoring. Collaborate with backend and platform engineers to define data contracts and interfaces.

Mentor engineers in best practices for data engineering and modelling. Essential Criteria: Strong experience building production-grade data pipelines at scale. Deep understanding of data modelling (warehouse, lake, semantic layers).

Experience with distributed data processing and orchestration frameworks. Strong SQL and data transformation skills. Experience designing data quality frameworks and validation processes.

Understanding of data architecture patterns (batch vs streaming, medallion architecture, etc.). Strong debugging and performance optimization skills. Experience working in cloud-native data environments (preferably AWS, but other major cloud platforms are acceptable).

Experience designing testing strategies for data pipelines. Desirable Criteria: Experience with AWS Glue, Spark, or similar distributed processing tools. Experience with data workflow orchestration tools (e.g., Temporal, Airflow).

Familiarity with financial or regulatory datasets – financial trading data experience will be a particular advantage. Exposure to semantic modelling, governance, or metadata management. What We Offer: A chance to work on genuinely interesting and important problems at scale.

A collaborative, supportive culture that values curiosity, integrity, and technical excellence. The opportunity to influence architecture, tooling, and future direction. A team of smart, humble, mission-driven people who care about our goals and about each other.

A modern, cloud-native tech stack and the freedom to bring your engineering voice to the table.