Senior Engineer- Data Engineering (ETL)
About the role
The Senior Engineer- Data Engineering role involves designing and developing data solutions for a major global corporation in the food marketing and distribution sector. The position requires implementing distributed, scalable data processing applications, adhering to CI/CD and DevOps principles, and ensuring high code quality through software engineering best practices. The engineer will work collaboratively in a cross-functional Agile team and participate in the full software lifecycle, from development to post-production support. Candidates should have a strong programming background in data operations, experience with large data volumes, relational/NoSQL databases, ETL tools, and CI/CD tools. The role also values strong communication skills and a desire for continuous learning.
Must have
- A Bachelor’s Degree in Computer Science or equivalent, and 3-5 years of experience in developing enterprise grade data processing applications
- A strong programming background in data ops (Python, Shell, SQL)
- Experience in processing large volumes of data
- Hands-on experience in working with relational/NoSQL databases and distributed storage engines (HDFS, S3, Redshift)
- Hands-on experience in ETL design and development using ETL tools (Preferably Informatica and cloud tools such as AWS Data Pipelines, Glue, Lambda, EMR, Spark, Hive)
- Working experience in a Scrum Agile delivery environment and with DevOps practices
- Experience in code management and CI/CD tools such as Github, Gitlab, Jenkins
- Experience in an agile environment and aligning Pod members on technical vision and path to implementation
- A strong desire to continue to grow your skillset
- Strong communication skills that are influential and convincing
Nice to have
- Experience in working with streaming data (using tools such as Kenisis, Kafka, Storm, Spark) will be an added advantage
- Experience in API development and user interfaces and related tools (NodeJS, AngularJS, HTML) will be an added advantage
Technologies
- Agile
- Angular
- AWS
- Bash
- CI/CD
- Data Engineering
- Data Pipelines
- DevOps
- ETL
- GitHub
- GitLab
- Hive
- HTML
- Jenkins
- Kafka
- AWS Lambda
- Node.js
- NoSQL
- Python
- Redshift
- S3
- Scrum
- Spark
- SQL