Technical Lead - Data Engineering
About the role
The Technical Lead - Data Engineering will be responsible for designing and developing large-scale data processing solutions for a major global corporation in the food marketing and distribution sector. The role involves collaborating with agile, cross-functional teams, providing guidance on database design and optimization, and ensuring adherence to DevOps principles. The position also includes designing scalability plans, maintaining high code quality through software engineering best practices, and participating in the full software lifecycle from development to post-production support. Candidates should have significant experience in enterprise application development, data integration, and team management, as well as hands-on expertise with large data volumes, distributed processing frameworks, ETL development, and relational databases.
Must have
- A bachelor's degree in computer science or equivalent, and 5/6+ years ofexperience developing production enterprise applications, data integration solutions and in managing teams.
- Excellent communication and leadership skills.
- Hands on experience working with large volumes of data and distributed processing frameworks (preferably Apache Spark).
- Hands-on experience in designand development of ETLs to process large volumes of data.
- Hands-on experience in workingwith large relational databases (preferably MySql).
- In-depth understanding of the database design/optimization methodologies (OLAP/OLTP database design techniques, query plan analysis).
- In-depth understanding of the software engineering methodologies (UML/Design patterns).
Nice to have
- Experience in working with NoSql databasesand bigdata (Hadoop,Elasticsearch, CouchDB, MongoDB) technologies will be an added advantage.
- Working in a Scrum Agile delivery environment and DevOps practices, as well as prior experience working with Cloud IaaS or PaaS providers such as AWS will be an added advantage.
Technologies
- Agile
- AWS
- CI/CD
- CouchDB
- Data Engineering
- DevOps
- Elasticsearch
- ETL
- Hadoop
- MongoDB
- MySQL
- NoSQL
- Scrum
- Spark
- UML
Jobs across Sri Lanka, watched for you.Colombo, Sri Lanka