Technical Lead

Technical Lead Location: Pune Experience: 8+yrs Work Model: Work from Office We are looking for a highly skilled Technical Lead (individual contributor) who has experience supporting and guiding a team of 7–8 engineers by solving complex technical challenges in a production support environment. The ideal candidate must be proactive in identifying opportunities for system improvements, performance enhancements, and automation across our data ecosystem. Key Responsibilities Act as the primary technical expert, supporting a team of 7–8 members by resolving complex Big Data and data engineering issues.

Work in a 24/7 or high-availability production support environment , ensuring stability, scalability, and performance of Big Data platforms. Troubleshoot performance bottlenecks across Spark, Hadoop, EMR, SQL queries, and data pipelines. Identify, propose, and implement improvements in data pipelines, architecture, performance tuning, and cost optimization.

Collaborate with cross-functional stakeholders (Data Engineering, DevOps, Architecture, etc.) to deliver scalable data solutions. Own incident management, root cause analysis (RCA), preventive actions, and optimization initiatives. Develop and review SQL queries, ETL logic, and data transformation code.

Ensure adherence to best practices in data quality, reliability, and security. Monitor and optimize workloads running on AWS EMR, Redshift, and other Big Data components. Drive automation to reduce manual intervention and improve operational efficiency.

Required Skills & Experience 7–10 years of total experience in Big Data technologies, with strong hands-on technical leadership. Prior experience supporting or guiding a 7–8 member team on technical problem-solving (IC role, not people management). Strong experience with AWS EMR, Redshift, Hadoop ecosystem , and distributed computing frameworks.

Expertise in Spark SQL, Hive, Presto , ETL workflows, and performance tuning. Must have excellent SQL development skills with the ability to write complex and optimized queries. Strong knowledge of job orchestration, scheduling, monitoring, and log analysis.

Experience working in a production support or DevOps-oriented environment. Good understanding of data modeling, partitioning, and optimization strategies. Ability to diagnose issues across large-scale data pipelines and distributed systems.

Good to Have Experience with Python, Airflow, or similar orchestration tools. Knowledge of AWS cost optimization for Big Data workloads. Familiarity with CI/CD pipelines and automation tools.

Exposure to data migration or modernization projects.