Data Engineer

Key Responsibilities and Duties

• Formulates system scope through research and fact-finding to develop or modify moderately complex information systems.
• Evaluates and implements changes to existing system logic difficulties and documentation as necessary.
• Reviews flow charts, models, and procedures and feasibility studies to design possible system solutions.
• Reviews technical documentation to guide system users and to assist with the ongoing operation, maintenance, and development of the system.
• Collaborates with system analysts, engineers, and programmers to design systems and to determine project limitations and capabilities, performance requirements, systems performance standards, and interfaces.
• Identifies and documents all system constraints, implications and consequences of various proposed system changes.
• Educates end users on system designs and functionalities through creation of training materials and conducting demos.
• Evaluates the efficiency and effectiveness of application operations and troubleshooting problems.

Skills: Data Warehouse - Must Have - Snowflake, AWS Glue, Python, Pyspark.

Build and maintain data pipelines using AWS Glue and Python • Develop SQL-based transformations loading data into Snowflake data warehouse • Implement data quality checks and validation rules • Support production data pipeline monitoring and issue resolution • Document data workflows, processes, and technical specifications

Technical Execution :

Write clean, maintainable Python code following team standards • Execute ETL/ELT processes extracting data from multiple sources • Perform unit testing and participate in code reviews • Use Terraform for infrastructure provisioning under supervision • Troubleshoot and fix data pipeline issues with guidance

Technical Skills :

Proficiency in Python programming • Strong SQL skills and database knowledge • Familiarity with AWS services (Glue, S3) or willingness to learn • Good understanding of Snowflake or similar data warehouse platforms • Knowledge of version control (Git) and collaboration tools • Understanding of data modeling concepts