Engineer - Data Engineering & Analytics
About the role
The Engineer - Data Engineering & Analytics role involves implementing real-time ETL/ELT pipelines on enterprise data platforms such as Databricks, Snowflake, Microsoft Fabric, Cloudera, Informatica IDMC, or Oracle. The position requires building data workflows, implementing Change Data Capture and streaming ingestion, applying dimensional data modelling, and ensuring data governance, security, and quality. The engineer will also apply DataOps practices, monitor and optimize production pipelines, and work directly with client stakeholders throughout the delivery lifecycle. Candidates should have a relevant degree and professional experience in building and operating enterprise data pipelines, with hands-on experience in at least two major data platforms and strong skills in Apache Spark and distributed data processing.
Must have
- Possess a Bachelor's Degree in Data Science or a higher qualification, such as an MSc in Data Science, Data Engineering, or Artificial Intelligence, from a recognised university
- Have 2–3 years of professional experience in building and operating enterprise data pipelines, data warehouses, or lakehouses
- Possess hands-on experience with at least two of the following platforms: Databricks, Snowflake, Microsoft Fabric/Azure Data Services, Cloudera, Informatica (IDMC/PowerCenter), or Oracle (ADW/Exadata/ODI)
- Demonstrate strong experience with Apache Spark and distributed data processing at scale
- Possess a solid understanding of data modelling, data quality, and data governance principles
- Demonstrate strong communication skills and the ability to work directly with client stakeholders
Nice to have
- Professional certifications such as Databricks Certified Data Engineer (Associate/Professional), SnowPro Core or SnowPro Advanced: Data Engineer, Microsoft Certified: Fabric Data Engineer Associate (DP-700) or Fabric Analytics Engineer Associate (DP-600), Informatica IDMC, or Oracle Autonomous Database certifications will be considered an added advantage
- Experience in migrating legacy ETL platforms, including Informatica PowerCenter, SSIS, or ODI, or on-premises data warehouses to modern cloud lakehouse platforms will be considered an added advantage
- Strong SQL and Python (PySpark) skills, along with knowledge of Scala or Java, will be considered an added advantage
Technologies
- Airflow
- Azure
- Azure Data Factory
- CI/CD
- Data Engineering
- Data Modeling
- Data Pipelines
- Data Science
- Data Warehousing
- Databricks
- ELT
- ETL
- Git
- Infrastructure as Code
- Java
- Kafka
- Lakehouse
- Microsoft Fabric
- Oracle
- Microsoft Purview
- PySpark
- Python
- Scala
- Snowflake
- Spark
- SQL
- SSIS
- Test Automation
- Uat
- Unity
Jobs across Sri Lanka, watched for you.Colombo, Sri Lanka