Kafka Admin

Job Title: Senior Big Data / Hadoop / Kafka / DevOps Administrator

Location: Denver, CO | St. Louis, MO also accepted. In-person interview is mandatory at the selected work location.

The key skills we should focus on are:

• Strong Terraform experience
• CI/CD expertise (any major toolset/platform is fine)
• Kafka experience any flavor, preferably with Confluent and Amazon MSK

Key Responsibilities:

Kafka Administration

• Deploy, configure, and manage Apache Kafka clusters and AWS MSK
• environments.
• Monitor broker health, partitions, replication factors, and consumer lag.
• Perform capacity planning and cluster scaling activities.
• Manage Kafka security using SSL, SASL, ACLs, and encryption standards.
• Troubleshoot producer, consumer, and broker performance issues.
• Support Kafka Connect, Schema Registry, Cruise Control, and MirrorMaker
• implementations.

AWS Cloud Administration

• Manage cloud infrastructure services including EC2, S3, IAM, VPC, EBS,
• CloudWatch, CloudTrail and AWS Glue.
• Support AWS Managed Streaming for Kafka (MSK), EMR, Lambda, and Airflow
• environments.
• Implement cloud security best practices and governance controls.
• Perform infrastructure provisioning and automation using Infrastructure as Code
• (IaC).
• Monitor cloud resource utilization and optimize operational costs.
• DevOps & Automation
• Design and maintain CI/CD pipelines using Jenkins, GitHub Actions, GitLab CI,
• or similar tools.
• Automate infrastructure deployment using Terraform, CloudFormation, and
• Ansible.
• Manage source control repositories and release processes.
• Implement monitoring and alerting solutions using Prometheus, Grafana, Splunk,
• ELK, or CloudWatch.
• Support containerization technologies such as Docker and Kubernetes.
• Develop automation scripts using Python, Shell, or Bash.

Operations & Support

• Provide Level 2 and Level 3 production support.
• Participate in on-call support rotations and incident management activities.
• Perform root cause analysis (RCA) and implement preventive measures.
• Create and maintain operational documentation and standard operating
• procedures.
• Ensure compliance with security, audit, and regulatory requirements.

Required Skills

• Apache Kafka Administration
• AWS Cloud Services
• Linux (RHEL/Rocky Linux)
• Shell Scripting and Python
• Jenkins, Git, Ansible, Terraform
• Docker and Kubernetes
• Monitoring Tools (Grafana, Prometheus, Splunk)
• Networking, Security, and High Availability Concepts
• Performance Tuning and Capacity Planning

Preferred Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or related field.
• Experience with Cloudera CDP, AWS MSK, Airflow, and Spark.
• AWS, GCP, Kafka, or Kubernetes certifications.
• Experience supporting large-scale production environments handling petabyte-
• scale data workloads.

Key Achievements Expected

• Maintain platform availability above 99.9%.
• Adopt AI-assisted engineering practices to improve operational efficiency,
• reduce manual effort, and accelerate troubleshooting and documentation.
• Automate repetitive operational tasks.
• Improve cluster performance and resource utilization.
• Ensure secure, scalable, and reliable data platform operations.
• Support enterprise data engineering, analytics, and AI/ML workloads efficiently.