Senior Consultant- Databricks
About the role
The Senior Consultant – Databricks at John Keells Information Technology (JKIT) will be responsible for designing, developing, deploying, and supporting scalable batch and streaming data pipelines using Azure Databricks, Spark, PySpark, Python, and SQL. The role involves implementing Lakehouse solutions, developing ETL/ELT frameworks, applying best practices in data modelling and governance, optimizing Spark workloads, ensuring security compliance, maintaining CI/CD pipelines, and collaborating with various stakeholders. Candidates should have a strong background in data engineering, hands-on experience with Databricks, and the ability to deliver high-quality, production-ready solutions. The company offers a collaborative environment focused on innovation and digital transformation. Applications should be sent via email as specified in the advert.
Must have
- A Bachelor's degree in Computer Science, Data Science, or a related discipline.
- 5+ years of experience in Data Engineering, Analytics, or software engineering, including 2+ years of enterprise Databricks experience.
- Strong expertise in Spark, PySpark, Python, SQL, Delta Lake, Databricks SQL, Notebooks, Workflows, and Repos.
- Experience with Azure Databricks, Data Factory, Data Lake Storage, Synapse, Microsoft Fabric, and Power BI.
- Experience with CI/CD, Git, automated testing, and release management.
- Databricks Professional certification is mandatory. Developer or Project Expert certification is an added advantage.
- Strong analytical, communication, documentation, customer engagement, and teamwork skills, with the ability to work independently and mentor junior engineers in Agile environments.
- Exposure to Asset Bundles, IAC, MLflow, MLOps, GenAI, Power BI, data observability, event streaming, presales, estimations, POCs, and customer demonstrations.
- Ability to deliver secure, scalable, production-ready solutions while meeting quality, performance, and cost targets and contributing to reusable assets and engineering standards.
Nice to have
- Azure/Fabric certifications are desirable.
Technologies
- Agile
- Azure
- CI/CD
- Data Engineering
- Data Lake
- Data Modeling
- Data Pipelines
- Data Science
- Databricks
- Delta Lake
- ELT
- ETL
- Generative AI
- Git
- Infrastructure as Code
- Lakehouse
- Microsoft Fabric
- MLflow
- MLOps
- Observability
- Power BI
- PySpark
- Python
- Spark
- SQL
- Test Automation
Highlights
Applicants must state the position in the subject line of their email when applying. The company is an equal opportunity employer and encourages applications from all suitably qualified individuals.
Jobs across Sri Lanka, watched for you.Colombo, Sri Lanka