GCP Platform Engineer

About the Company One of the active BFSI clients is looking for immediate hiring, company is based in USA and footprints in Chennai and Hyderabad We are hiring a GCP Platform Engineering Lead who will design, configure, implement, and scale the enterprise data platform on GCP using Dataproc, GCS, Dataflow and Composer. This role is L3-level technical ownership, responsible for building platform components, defining standards, enabling developers, and troubleshooting platform issues. The right candidate will be an engineering builder – someone who automates, designs frameworks, and drives platform maturity, not operations support.

About the Role This role involves key responsibilities in platform build and engineering, troubleshooting, developer enablement, automation, CI/CD, tooling, performance, reliability, and platform evolution. Responsibilities Platform Build & Engineering: Architect, deploy, and configure GCP data platform components (Dataproc clusters, Composer environments, Dataflow workers, GCS storage controls, IAM policies). Build Terraform-driven reusable templates and configuration blueprints for platform provisioning.

Implement security & governance controls – CMEK, VPC Service Controls, org policies, workload identity, IAM least privilege. Define platform standards – autoscaling rules, cluster profiles, dependency packaging, environment provisioning workflows. L3 Troubleshooting & Developer Enablement: Serve as platform L3 escalation for engineering teams to debug Dataproc/YARN execution failures, Dataflow backlog or performance bottlenecks, Composer scheduler or DAG dependency errors.

Provide reference architectures, runbooks, onboarding documentation. Enable self-service platform usage via templates, automation and tooling. Automation, CI/CD & Tooling: Build internal automation to provision environments using IaC and scripting.

Integrate CI/CD pipelines (Cloud Build, GitHub Actions) for Composer DAGs, Dataflow templates and platform configurations. Develop frameworks for dependency packaging, deployment workflows and observability. Performance, Reliability & Platform Evolution: Monitor platform health using Cloud Monitoring, tune performance, and scale capacity.

Optimize cost across clusters/jobs, introduce budget controls and quotas. Participate in platform roadmap, architecture modernization and platform upgrades. Qualifications Core GCP Data Services Expertise: Dataproc – cluster config, init-actions, HA setups, autoscaling, debugging YARN.

GCS – IAM policy design, lifecycle management, CMEK, cross-project sharing. Dataflow – Flex Templates, worker sizing, backlog handling, tuning pipelines. Composer – Airflow deployments, dependency management, scaling, DAG troubleshooting.

Required Skills Engineering Skills: Terraform (preferred) or IaC experience. Python or Shell scripting for automation. CI/CD integration for platform workloads.

Spark/Hadoop understanding for job failure troubleshooting. Leadership & Collaboration: Ability to lead platform roadmap execution while remaining hands-on. Strong collaboration and communication skills across architecture, security & engineering teams.

Ownership mindset and documentation capability. Preferred Skills Preferred Certifications: Google Professional Data Engineer Google Cloud Architect Google Associate Cloud Engineer