Senior Engineer – Data Engineering & Analytics
About the role
The Senior Engineer – Data Engineering & Analytics is responsible for designing, building, and maintaining scalable batch and real-time ETL/ELT pipelines on various enterprise data platforms. The role involves developing lakehouse and data warehouse solutions, orchestrating data workflows, implementing CDC and streaming ingestion, applying dimensional data modelling, and developing Power BI semantic models. The engineer will also implement data governance, prepare data for AI/ML use cases, apply DataOps practices, monitor and optimize production pipelines, work directly with client stakeholders, and mentor junior engineers. The position requires a strong background in data engineering, hands-on experience with major data platforms, and excellent communication skills.
Must have
- Possess a Bachelor's Degree in Data Science or a higher qualification, such as an MSc in Data Science, Data Engineering, or Artificial Intelligence, from a recognized university
- Have 2–5 years of professional experience in building and operating enterprise data pipelines, data warehouses, or lakehouses
- Possess hands-on experience with at least two of the following platforms: Databricks, Snowflake, Microsoft Fabric/Azure Data Services, Cloudera, Informatica (IDMC/PowerCenter), or Oracle (ADW/Exadata/ODI)
- Demonstrate strong experience with Apache Spark and distributed data processing at scale
- Possess a solid understanding of data modelling, data quality, and data governance principles
- Have experience developing Power BI semantic models and reports
- Demonstrate strong communication skills and the ability to work directly with client stakeholders
- Demonstrate strong SQL and Python (PySpark) skills
Nice to have
- Professional certifications such as Databricks Certified Data Engineer (Associate/Professional), SnowPro Core or SnowPro Advanced: Data Engineer, Microsoft Certified: Fabric Data Engineer Associate (DP-700) or Fabric Analytics Engineer Associate (DP-600), Informatica IDMC, or Oracle Autonomous Database certifications will be considered an added advantage
- Experience in migrating legacy ETL platforms such as Informatica PowerCenter, SSIS, or ODI, or on-premises data warehouses to modern cloud lakehouse platforms will be considered an added advantage
- Experience with real-time streaming and CDC tools, including Kafka and Oracle GoldenGate, will be considered an added advantage
- Exposure to GenAI data engineering, including RAG pipelines, vector databases, and LLM-ready data preparation, will be considered an added advantage
- Experience with dbt, Terraform, Azure DevOps, or GitHub Actions will be considered an added advantage
- Prior experience in banking, telecommunications, or public-sector data projects will be considered an added advantage
- knowledge of Scala or Java will be considered an added advantage
Technologies
- Airflow
- Azure
- Azure Data Factory
- Azure DevOps
- CI/CD
- Data Engineering
- Data Modeling
- Data Pipelines
- Data Science
- Data Warehousing
- Databricks
- dbt
- Delta Lake
- ELT
- ETL
- Generative AI
- Git
- GitHub
- GitHub Actions
- Iceberg
- Infrastructure as Code
- Java
- Kafka
- Lakehouse
- LLM
- Machine Learning
- Microsoft Fabric
- Oracle
- Power BI
- Microsoft Purview
- PySpark
- Python
- RAG
- Scala
- Snowflake
- Spark
- SQL
- SSIS
- Terraform
- Test Automation
- Uat
- Unity
- Vector Databases
- Vector Search
Jobs across Sri Lanka, watched for you.Colombo, Sri Lanka