Overview
Build and run batch and streaming data pipelines on modern data platforms.
What you'll do
- Build batch and streaming pipelines into warehouses and lakehouses
- Model data for analytics and reporting
- Ingest data from enterprise source systems
- Build data quality checks and monitor pipelines
- Orchestrate, test and run CI/CD for data pipelines
- Optimise performance and cost
- Work with analysts, data scientists and architects
What you bring
- Hands-on data engineering experience
- Python and SQL
- Spark or PySpark
- Databricks, Snowflake or a similar platform
- Orchestration tools such as Airflow
- Cloud data services on AWS or Azure
- Version control and CI/CD practices
You don't need every item on this list. If most of it fits, send your CV.
Nice to have
- dbt
- Kafka or other streaming technology
- Industry data domain experience
- Infrastructure as code
- Data governance tooling
- Platform certifications