Senior Data Engineer

Hello,

Job opening at Ai Palette-Bengaluru, please find job details below :-

Job Title – Senior Data Engineer

Experience Level: 5-8 years

Location: Bangalore

Key Responsibilities

• Build and maintain Python-based ETL pipelines and data automation scripts using pandas and numpy
• Write and optimize SQL queries in PostgreSQL, support schema changes, and maintain data integrity
• Develop and maintain Apache Airflow DAGs for batch scheduling and pipeline orchestration
• Build data validation and quality check scripts to ensure pipeline reliability
• Support Elasticsearch index maintenance, data ingestion, and basic mapping updates
• Integrate LLM-generated outputs into structured data pipelines and storage systems
• Write unit tests for data pipelines and instrument code for observability
• Guide junior engineers on data pipeline initiatives
• Collaborate with the product team to design and improve the data layer

Key Qualifications

• 4-8 years of experience in data engineering, backend engineering, or a related technical role
• Python (strong): comfortable writing production scripts, proficient with pandas and numpy for data processing and transformation
• SQL (solid): can write complex queries, understands joins, aggregations, window functions, and indexing basics
• PostgreSQL experience: basic schema understanding, data types, JSONB familiarity is a plus
• Experience building automation scripts for data processing, validation, or reporting
• Familiarity with at least one orchestration tool (Airflow, Prefect, or similar)
• Git, Docker, and standard development workflows
• Writes test cases as part of development

Thanks & Regards,

Salim

(Human Resources)