Data Engineer
Optomi
Job Description
Design and implement scalable batch and near-real-time data pipelines. Develop ETL/ELT workflows optimized for performance and cost. Implement dimensional data models and standardize business metrics.
Instrument APIs and user journeys to capture behavioral and transactional data. Data Governance & Quality Ensure data integrity , governance, privacy , and compliance. Maintain reliability and availability of mission-critical systems.
ML & Advanced Use Cases Enable RAG-based data preparation and AI-driven automation. Required Qualifications 6+ years of experience in data engineering for analytics or ML systems. Experience in Python, Scala, or Java.
Hands-on experience with Spark, Kafka, and Airflow (or similar). Strong understanding of data modeling and lakehouse architectures (e.g., Iceberg). Experience with AWS, Azure, or GCP .
Experience with Snowflake, Databricks, Trino, OLAP/NRT systems, Superset or Tableau. Familiarity with CI/CD, data observability , infrastructure-as-code. Exposure to MLOps and GenAI/RAG pipelines.
Hands-on experience with LLMs (prompt engineering, fine-tuning, RAG). Experience in FinTech, Wallet, or Payments domain. Nice-to-haves: Docker, Kubernetes, Splunk, Grafana, Scala, GitLab, Spinnaker, Datadog, Rust, GO, or MLOps/GenAI experience. #J-18808-Ljbffr