Automation Specialist

About us:

We're a leading provider of comparison services for insurance, utilities, and personal finance products.

Our mission is to make Australians' lives easier by saving them time, money, and effort.

The Data Automation Specialist will be responsible for automating, monitoring, and optimizing data pipelines and infrastructure to ensure efficient, scalable, and reliable data workflows. This role focuses on best practices, CI/CD automation, cloud infrastructure, observability, and security.

• Streamlining data pipeline deployment through CI/CD
• Ensuring observability, performance, and security in cloud data environments
• Automating infrastructure management and data workflow orchestration
• Driving best practices for governance and cost optimization

This role requires expertise in DevOps, cloud data platforms, infrastructure automation, and data pipeline orchestration. The ideal candidate will work closely with Data Engineers, Data Scientists, Platform Engineers, and Security teams to improve data availability, governance, and operational efficiency.

Data Pipeline Automation & Orchestration:

• Design, implement, and maintain CI/CD pipelines for automated data workflow deployment
• Develop infrastructure-as-code (IaC) templates for provisioning cloud-based data environments/resources
• Automate ETL/ELT workflow management using orchestration tools like Apache Airflow, Prefect, or AWS Step Functions
• Optimize data ingestion, transformation, and processing pipelines for scalability and efficiency

Cloud Infrastructure & Security:

• Manage data infrastructure in AWS, Azure, or GCP, ensuring high availability and security
• Implement role-based access control (RBAC), encryption, and data security best practices
• Collaborate with Security teams to ensure compliance with industry regulations
• Monitor cloud costs and optimize resource usage for storage, compute, and networking

Monitoring, Observability & Incident Response:

• Develop real-time monitoring dashboards and alerts for data pipeline performance and failures
• Use observability tools like Datadog, Prometheus, Grafana, CloudWatch etc.to track latency, errors, and throughput
• Establish incident response workflows and automated rollback strategies for failures
• Work with Site Reliability Engineers (SRE) and DevOps teams to maintain high system uptime and resilience

DataOps & DevOps Best Practices:

• Implement version control, automated testing, and release management for data pipelines
• Promote methodologies to improve collaboration between Data Engineering, IT, and Business teams
• Ensure proper data governance, metadata tracking, and auditability in data workflows
• Optimize data processing efficiency

Candidate Requirements:

Qualifications: Bachelor's or Master's degree in Computer Science, Data Engineering, Cloud Computing, or a related field/relevant experience.

Experience & Skills:

• 3+ years of experience in DevOps, Data Engineering, or Cloud Infrastructure roles
• Experience with CI/CD tools (GitHub Actions, Jenkins, Azure DevOps etc.)
• Proficiency in orchestration tools (Apache Airflow, Prefect, Dagster, AWS Step Functions etc.)
• Strong understanding of ETL/ELT pipelines, workflow scheduling, and automation
• Hands-on experience with cloud data platforms (AWS, Azure, GCP)
• Proficiency in Terraform for infrastructure-as-code (IaC)
• Experience with observability tools (Datadog, Prometheus, Grafana, CloudWatch, Splunk etc.)
• Strong troubleshooting skills for data pipeline failures, latency, and performance optimization
• Proficiency in Python, Bash, or PowerShell for automation and scripting
• Experience with SQL and NoSQL databases (PostgreSQL, Snowflake, BigQuery, DynamoDB, etc.)
• Ability to work with cross-functional teams (Data Engineering, IT, Security, Business Intelligence)
• Strong documentation skills to maintain runbooks, architecture diagrams, and pipeline workflows

What's in it for you:

A competitive salary plus generous commission scheme

The opportunity to work with a well-known household brand

Work with a team of highly energized professionals

Recharge days per year

A free coffee each day

An impressive Employee Benefits Program

If you are someone who fosters team spirit, has resilience, and aspires to continue learning and developing your skills to drive commercial outcomes, you will have tremendous success in helping us achieve our progressive growth agendas. High energy levels and a love of having fun are also important to working in this business.