Data Pipeline Architect

2 days ago

Brisbane, Queensland, Australia beBeeData Full time $100,000 - $140,000
Job Title: Data Pipeline Architect

This role involves the design, implementation and maintenance of data pipelines that are efficient, scalable and reliable. The ideal candidate will have expertise in DevOps, cloud data platforms, infrastructure automation and data pipeline orchestration.

The primary objectives of this role are:

  • Streamlining data pipeline deployment through continuous integration and delivery (CI/CD)
  • Ensuring observability, performance and security in cloud data environments
  • Automating infrastructure management and data workflow orchestration
  • Driving best practices for data engineering, governance and cost optimization

The key responsibilities of this role include:

Data Pipeline Automation & Orchestration
  • Designing, implementing and maintaining CI/CD pipelines for automated data workflow deployment
  • Developing infrastructure-as-code (IaC) templates for provisioning cloud-based data environments/resources
  • Automating ETL/ELT workflow management using orchestration tools like Apache Airflow, Prefect or AWS Step Functions
  • Optimizing data ingestion, transformation and processing pipelines for scalability and efficiency

Cloud Infrastructure & Security

  • Managing data infrastructure in AWS, Azure or GCP, ensuring high availability and security
  • Implementing role-based access control (RBAC), encryption and data security best practices
  • Collaborating with security teams to ensure compliance with industry regulations (GDPR, HIPAA, SOC2)
  • Monitoring cloud costs and optimizing resource usage for storage, compute and networking

Monitoring, Observability & Incident Response

  • Developing real-time monitoring dashboards and alerts for data pipeline performance and failures
  • Using observability tools like Datadog, Prometheus, Grafana or CloudWatch to track latency, errors and throughput
  • Establishing incident response workflows and automated rollback strategies for failures
  • Working with Site Reliability Engineers (SRE) and DevOps teams to maintain high system uptime and resilience

Key Skills and Qualifications

  • 3+ years of experience in DevOps, data engineering or cloud infrastructure roles
  • Experience with CI/CD tools like GitHub Actions, Jenkins or Azure DevOps
  • Proficiency in orchestration tools like Apache Airflow, Prefect or AWS Step Functions
  • Strong understanding of ETL/ELT pipelines, workflow scheduling and automation
  • Hands-on experience with cloud data platforms like AWS, Azure or GCP
  • Proficiency in Terraform for infrastructure-as-code (IaC)
  • Experience with observability tools like Datadog, Prometheus, Grafana or CloudWatch
  • Strong troubleshooting skills for data pipeline failures, latency and performance optimization
  • Proficiency in Python, Bash or PowerShell for automation and scripting
  • Experience with SQL and NoSQL databases like PostgreSQL, Snowflake, BigQuery or DynamoDB

Benefits

  • Competitive salary plus generous uncapped commission scheme
  • Opportunity to work with a well-known brand and public listed company
  • Work with a team of highly energized, motivated and driven professionals
  • 3 'recharge' days per year (yes, in addition to your annual leave)
  • Free coffee each day, funky café and state-of-the-art space
  • Impressive Employee Benefits Program

  • Enterprise Data Architect

    2 days ago

    Brisbane, Queensland, Australia beBeeData Architect Full time $100,000 - $135,000

    About this RoleWe are seeking a seasoned enterprise data architect to develop and maintain robust data models that support business strategy.The ideal candidate will possess in-depth knowledge of SAP HANA 2.0, including data modelling, calculation views, security roles, and data quality tools.Responsibilities include:Designing, developing, and maintaining...

  • Data Pipeline Architect

    4 days ago

    Brisbane, Queensland, Australia beBeeDataAutomation Full time $100,000 - $150,000

    Senior Data Automation SpecialistAbout the Role:We are seeking a highly skilled Senior Data Automation Specialist to lead our data operations team. This role is responsible for designing, implementing, and maintaining automated data workflows and infrastructure to ensure efficient, scalable, and reliable data delivery.Key Responsibilities:Design and...

  • Analytical Data Pipeline Specialist

    2 days ago

    Brisbane, Queensland, Australia beBeeDataEngineer Full time $140,000 - $180,000

    Senior Data Engineer Job DescriptionAre you a skilled data engineer looking for a new challenge? Do you have experience building analytical data pipelines in cloud systems? We are seeking a Senior Data Engineer to join our team and help us drive value for the business.The ideal candidate will have solid experience building ETL pipelines for production...

  • Expert Data Architect Required

    3 days ago

    Brisbane, Queensland, Australia beBeeDataArchitect Full time $147,000 - $183,000

    Job DescriptionWe are seeking an expert Data Architect to lead the design, development, and optimization of our enterprise data platform. As a key member of the data team, you will shape the data strategy, ensuring scalability, reliability, and performance across the data ecosystem.You will architect and lead the development of scalable, secure, and...

  • Enterprise Architect

    24 hours ago

    Brisbane, Queensland, Australia Data#3 Full time

    Data#3 Brisbane City, Queensland, Australia Join or sign in to find your next job Join to apply for the Enterprise Architect role at Data#3 Data#3 Brisbane City, Queensland, Australia2 days ago Be among the first 25 applicants Join to apply for the Enterprise Architect role at Data#3 Direct message the job poster from Data#34-month contract | Brisbane CBD |...

  • Enterprise Architect

    1 week ago

    Brisbane, Queensland, Australia Data#3 Full time

    Data#3 Brisbane City, Queensland, AustraliaJoin or sign in to find your next jobJoin to apply for the Enterprise Architect role at Data#3Data#3 Brisbane City, Queensland, Australia2 days ago Be among the first 25 applicantsJoin to apply for the Enterprise Architect role at Data#3Direct message the job poster from Data#34-month contract | Brisbane CBD | Qld...

  • Enterprise Architect

    7 days ago

    Brisbane, Queensland, Australia Data#3 Full time

    Data#3 Brisbane City, Queensland, AustraliaJoin or sign in to find your next jobJoin to apply for the Enterprise Architect role at Data#3Data#3 Brisbane City, Queensland, Australia2 days ago Be among the first 25 applicantsJoin to apply for the Enterprise Architect role at Data#3Direct message the job poster from Data#34-month contract | Brisbane CBD | Qld...

  • Azure Data Architect

    2 days ago

    Brisbane, Queensland, Australia beBeeData Full time $320,000 - $365,000

    Senior Data Solutions Architect">This is an exciting opportunity to work on high-impact data projects that shape the future of digital services in the public sector.About the RoleDesign, implement and maintain robust, scalable, secure and efficient data pipelinesOptimize SQL queries for performance tuning and database managementMonitor troubleshoot and...

  • Data Architect

    3 weeks ago

    Brisbane, Queensland, Australia Allens Full time

    Join to apply for the Data Architect role at Allens2 days ago Be among the first 25 applicants Join to apply for the Data Architect role at Allens Get AI-powered advice on this job and more exclusive features.Your role At Allens, our business teams are also specialists in their field, using their unique knowledge and expertise to solve critical client and...

  • Enterprise Architect

    22 hours ago

    Brisbane, Queensland, Australia Data#3 Full time

    Data#3 Brisbane City, Queensland, AustraliaJoin or sign in to find your next jobJoin to apply for theEnterprise Architectrole atData#3Data#3 Brisbane City, Queensland, Australia2 days ago Be among the first 25 applicantsJoin to apply for theEnterprise Architectrole atData#3Direct message the job poster from Data#34-month contract | Brisbane CBD | Qld...