Senior Cloud Data Engineer

Data Engineer Opportunity

We are seeking a highly skilled Data Engineer to join our team.

The ideal candidate has expertise in cloud data platforms, infrastructure automation, and data pipeline orchestration.

The primary objectives of the role are:

• Streamlining data pipeline deployment through continuous integration and delivery (CI/CD)
• Ensuring observability, performance, and security in cloud data environments
• Automating infrastructure management and data workflow orchestration
• Driving best practices for data governance and cost optimization

Data Pipeline Automation & Orchestration

• Design, implement, and maintain CI/CD pipelines for automated data workflow deployment
• Develop infrastructure-as-code (IaC) templates for provisioning cloud-based data environments/resources
• Automate Extract, Transform, Load (ETL)/Extract, Load, Transform (ELT) workflow management using orchestration tools like Apache Airflow, Prefect, or AWS Step Functions
• Optimize data ingestion, transformation, and processing pipelines for scalability and efficiency

Cloud Infrastructure & Security

• Manage data infrastructure in Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP), ensuring high availability and security
• Implement role-based access control (RBAC), encryption, and data security best practices
• Collaborate with Security teams to ensure compliance with industry regulations (General Data Protection Regulation (GDPR), Health Insurance Portability and Accountability Act (HIPAA), Service Organization Control 2 (SOC2))
• Monitor cloud costs and optimize resource usage for storage, compute, and networking

Monitoring, Observability & Incident Response

• Develop real-time monitoring dashboards and alerts for data pipeline performance and failures
• Use observability tools such as Datadog, Prometheus, Grafana, CloudWatch, etc. to track latency, errors, and throughput
• Establish incident response workflows and automated rollback strategies for failures
• Work with Site Reliability Engineers (SRE) and DevOps teams to maintain high system uptime and resilience

DataOps & DevOps Best Practices

• Implement version control, automated testing, and release management for data pipelines
• Promote DataOps methodologies to improve collaboration between Data Engineering, IT, and Business teams
• Ensure proper data governance, metadata tracking, and auditability in data workflows
• Optimize data processing efficiency

Candidate Requirements

Qualifications

• Bachelor's or Master's degree in Computer Science, Data Engineering, Cloud Computing, or a related field/relevant experience.

Experience & Skills

• 3+ years of experience in DevOps, Data Engineering, or Cloud Infrastructure roles
• Experience with CI/CD tools (GitHub Actions, Jenkins, Azure DevOps, etc.)
• Proficiency in orchestration tools (Apache Airflow, Prefect, Dagster, AWS Step Functions, etc.)
• Strong understanding of ETL/ELT pipelines, workflow scheduling, and automation
• Hands-on experience with cloud data platforms (AWS, Azure, GCP)
• Proficiency in Terraform for infrastructure-as-code (IaC)
• Experience with observability tools (Datadog, Prometheus, Grafana, CloudWatch, Splunk, etc.)
• Strong troubleshooting skills for data pipeline failures, latency, and performance optimization
• Proficiency in Python, Bash, or PowerShell for automation and scripting
• Experience with SQL and NoSQL databases (PostgreSQL, Snowflake, BigQuery, DynamoDB, etc.)
• Ability to work with cross-functional teams (Data Engineering, IT, Security, Business Intelligence)
• Strong documentation skills to maintain runbooks, architecture diagrams, and pipeline workflows

What We Offer

• A competitive salary plus generous uncapped commission scheme
• The opportunity to work with a well-known brand
• Work with a team of highly energized, motivated, and driven professionals
• Three recharge days per year (in addition to annual leave)
• A free coffee each day, a funky café, and a state-of-the-art space
• An impressive Employee Benefits Program

This is an exciting opportunity for a motivated individual who fosters team spirit, has resilience, and aspires to continue learning and developing their skills to drive commercial outcomes. High energy levels and a love of having fun are also important to working in this energetic business.