Senior Data Engineer
About the role
The Senior Data Engineer role is a contract position based in Colombo with a hybrid work arrangement. The position is with a technology and digital solutions company specializing in data, cloud, digital transformation, and enterprise technology. The role involves developing and supporting scalable data pipelines that connect enterprise source systems with MDM and governed downstream platforms, focusing on data ingestion, standardization, transformation, and data quality. Key responsibilities include building ingestion pipelines, developing PySpark/Spark transformations on Azure Databricks, implementing data patterns, and ensuring reliable data availability for downstream consumption.
Must have
- Overall 7+ years of experience and 5+ years data engineering with strong production PySpark/Spark experience.
- Strong Azure Databricks, Delta Lake, SQL and ADLS Gen2.
- REST/API ingestion and database integration experience.
- Incremental/CDC processing and schema-evolution experience.
- Strong data-quality/testing practices and Git/CI/CD.
- Experience handling large, messy enterprise datasets
- Good English communication skills, both written and verbal (minimum 7/10 proficiency)
Nice to have
- Databricks certification.
- MDM/entity-resolution project exposure.
- Azure Data Factory or equivalent orchestration.
- Exposure to ML-based entity resolution / fuzzy matching (PySpark MLlib), including deterministic and probabilistic matching design
- Purview/Unity Catalog governance exposure.
Technologies
- Azure
- Azure Data Factory
- CI/CD
- Data Engineering
- Data Ingestion
- Data Pipelines
- Databricks
- Delta Lake
- Git
- Machine Learning
- MDM
- PySpark
- REST
- Spark
- SQL
- Unity
Highlights
Applicants are instructed to send their CV to careers.talentstudio.io. The role requires hybrid work in Colombo. Databricks certification and experience with MDM/entity-resolution projects are considered advantageous.
Jobs across Sri Lanka, watched for you.Colombo, Sri Lanka