Apache Nifi Developer

Job Summary We are seeking a highly skilled Apache NiFi Developer to lead the design, development, and optimization of our enterprise-scale. In this role, you will build and maintain high-performance, real-time, and batch data pipelines that automate data flows across heterogeneous systems. You will focus on building robust ingestion, routing, and transformation frameworks while ensuring maximum system uptime, security, and scalability. Key Responsibilities Pipeline Development : Design, build, and optimize scalable Apache NiFi data flows using processors, controller services, and parameter contexts. System Integration : Connect diverse sources and sinks, including SQL/NoSQL databases, cloud storage (AWS S3, Azure Blob), and REST APIs. Streaming & Messaging : Integrate NiFi pipelines with Apache Kafka, Logstash, and Cloudera Data Platform (CDP). Performance Tuning : Manage cluster performance, configure back-pressure thresholds, tune JVM configurations, and resolve processing bottlenecks. Custom Extensibility : Write custom NiFi processors using Java or execute pipeline logic via Groovy/Python scripting. Data Governance : Maintain data provenance, enforce schema compliance (Avro, Parquet, JSON), and guarantee data lineage.

Lifecycle Management : Utilize NiFi Registry for version control, continuous integration, and seamless deployment across dev, staging, and production environments. Security & Compliance : Implement TLS encryption, Kerberos authentication, and Role-Based Access Control (RBAC). Operations & Monitoring : Set up production alerting tools like Prometheus and Grafana to proactively handle delivery failures.