Senior Data Engineer (Remote)
About the role
The Senior Data Engineer will be responsible for designing, developing, and optimizing scalable, reliable, and high-performance data platforms and pipelines. The role involves building and maintaining data warehouses, data marts, and scalable data models, as well as developing complex SQL solutions and scalable AWS Glue jobs using PySpark/Python. The engineer will also design and maintain data ingestion and replication pipelines, implement incremental loading and CDC strategies, and ensure data quality and performance optimization. The position requires strong hands-on experience with Snowflake, AWS Glue, PySpark/Python, SQL Server/T-SQL, Fivetran, and modern ETL/ELT architecture. The successful candidate will collaborate with various stakeholders to deliver reliable data solutions and contribute to technical architecture and engineering standards.
Must have
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, IT, Engineering, or a related field.
- 5+ years of professional experience in Data Engineering, ETL/ELT, Data Warehousing, or a related field, with senior-level responsibilities.
- Strong hands-on experience with Snowflake, including data modeling, SQL, performance optimization, security, and cost optimization.
- Strong hands-on experience with AWS Glue and PySpark/Python for production-grade data processing.
- Strong experience with SQL Server/T-SQL, including complex queries, stored procedures, CTEs, window functions, indexing, query optimization, and performance troubleshooting.
- Practical experience with Fivetran or similar data integration/replication platforms.
- Strong understanding of ETL/ELT architecture, data warehousing, dimensional modeling, data marts, incremental loading, CDC, and data pipeline design.
- Good knowledge of AWS data services including S3, IAM, CloudWatch, Lambda, and Secrets Manager.
- Experience with Git, CI/CD, code reviews, automated testing, monitoring, and production support.
- Strong problem-solving, analytical, communication, and stakeholder collaboration skills.
- Experience working in Agile/Scrum environments.
- Snowflake | AWS Glue | PySpark/Python | SQL Server/T-SQL | Fivetran | ETL/ELT | Data Warehousing | Data Modeling | Dimensional Modeling | CDC | Incremental Loading | AWS S3/IAM/CloudWatch
Nice to have
- Experience with Machine Learning model development, deployment, monitoring, and the end-to-end ML lifecycle.
- Experience with scikit-learn, TensorFlow, or similar ML frameworks.
- Experience integrating ML/data science workflows with cloud data platforms and production pipelines.
- Experience with Power BI, including DAX, Power Query, data modeling, and performance optimization.
- Experience developing Streamlit dashboards and data applications using Python.
- Understanding of how Data Engineering, BI, Analytics, and Machine Learning integrate within a modern data platform.
Technologies
- Agile
- AWS
- AWS Glue
- CI/CD
- CloudWatch
- Data Engineering
- Data Ingestion
- Data Modeling
- Data Pipelines
- Data Science
- Data Warehousing
- Dimensional Modeling
- ELT
- ETL
- Fivetran
- Git
- IAM
- AWS Lambda
- Machine Learning
- Power Query
- Power BI
- PySpark
- Python
- S3
- scikit-learn
- Scrum
- Snowflake
- SQL
- SQL Server
- Streamlit
- TensorFlow
- Test Automation
Highlights
This is a remote position. Applicants should send their CV to hello@aventude.com. Submitted resumes will be retained for 6 months. For questions about the application or data retention, contact hello@aventude.com.
Jobs across Sri Lanka, watched for you.Colombo, Sri Lanka