Date Engineer
Diverse Lynx
Job Description
Job title: Data Engineer Location: LOCAL TO Naperville, IL ONLY! Duration: Contract Keywords / Technologies : Big Data , Hadoop Ecosystem, Python for Data Science, IBM Data Management, Talend, DataStage, Spark, Hive, Kafka , ETL/ELT , SQL, NoSQL, Airflow. Job Description Key Responsibilities Design, develop, and maintain ETL/ELT pipelines using Talend and IBM DataStage. Build and optimize data pipelines and workflows for large-scale data processing. Work with Python (Pandas, PySpark) for data transformation, processing, and API integrations. Develop and maintain Big Data solutions using technologies such as Spark, Hadoop, Hive, and Kafka. Implement data engineering best practices and design patterns to ensure scalability, reliability, and performance. Write and optimize complex SQL queries and work with NoSQL databases . Manage and monitor data workflows using orchestration tools like Airflow, Control-M, or Talend Scheduler .
Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders. Ensure data quality, governance, and performance optimization across the data ecosystem.