Current jobs related to Chief Data Pipeline Architect - Melbourne, Victoria - beBeeCloudDataEngineer

  • Chief Cloud Data Architect

    2 weeks ago

    Melbourne, Victoria, Australia beBeeData Full time $150,000 - $180,000

    Cloud Data Architecture OpportunityThis is a high-impact digital program aimed at enhancing enterprise data capabilities across cloud platforms. As a Chief Cloud Data Architect, you will play a pivotal part in delivering high-performance data infrastructure using AWS, Snowflake, dbt, and Kafka.Main Responsibilities:Design and build scalable data pipelines...

  • Chief Data Architect

    2 weeks ago

    Melbourne, Victoria, Australia beBeeData Full time $140,000 - $180,000

    Job Opportunity: Data ExpertWe are developing cutting-edge AI models in Australia, from the ground up. Our foundation models necessitate large-scale datasets, designed and built by our team.As a Senior Data Engineer, you will spearhead the design and construction of these datasets, working hands-on to source, clean, transform, and structure massive amounts...

  • Chief Data Architect

    11 hours ago

    Melbourne, Victoria, Australia beBeeDataExpert Full time $190,000 - $220,000

    We are seeking a seasoned data architect to spearhead scalable data solutions across our organization.Key Responsibilities:Design, build and maintain high-performance data pipelines and ETL processes.Work with large datasets in cloud environments to deliver accurate, reliable data insights.Collaborate with cross-functional teams to support analytical and...

  • Chief Data Architect

    4 hours ago

    Melbourne, Victoria, Australia beBeeData Full time $170,000 - $190,000

    Enterprise Data Platform EngineerThis strategic opportunity enables you to craft and deploy large-scale data platforms, supporting core digital services for millions of users. You will be part of a mission-critical project that places data engineering at the heart of faster, smarter, and safer service delivery.Key Responsibilities:Design and maintain...

  • Data Pipeline Architect

    2 weeks ago

    Melbourne, Victoria, Australia beBeeEngineer Full time $120,000 - $150,000

    Strategic Data EngineerThis role involves designing, building and maintaining data pipelines that drive informed decision-making across the business.Key Responsibilities:Owning the end-to-end development of scalable, sustainable data pipelines in cloud-based environmentsOptimizing and fine-tuning our data infrastructure to ensure high-performance,...

  • Chief Data Platform Architect

    2 weeks ago

    Melbourne, Victoria, Australia beBeeDataEngineer Full time $170,000 - $190,000

    A major transformation program is underway to modernise legacy data systems using AWS Cloud and Databricks.As a key member of the project, you will have the opportunity to shape and deliver enterprise-scale data platforms supporting core digital services for millions of users.Key Responsibilities:Design and build scalable, secure data pipelines using...

  • Data Architect

    2 weeks ago

    Melbourne, Victoria, Australia TapTalent Full time

    We're Hiring: Data Architect We are seeking an experienced and innovative Data Architect to design and implement robust data solutions that drive business intelligence and analytics. The ideal candidate will have expertise in data modeling, database design, and enterprise data architecture to support our organization's data-driven decision making. Location:...

  • Data Architect

    2 weeks ago

    Melbourne, Victoria, Australia TapTalent Full time

    We're Hiring: Data Architect We are seeking an experienced and innovative Data Architect to design and implement robust data solutions that drive business intelligence and analytics. The ideal candidate will have expertise in data modeling, database design, and enterprise data architecture to support our organization's data-driven decision making. Location:...

  • Pipeline System Architect

    2 weeks ago

    Melbourne, Victoria, Australia beBeeSystemDesigner Full time $140,000 - $170,000

    Pipeline System ArchitectWe are seeking a highly skilled professional to fill the Pipeline System Architect role in Canberra, Melbourne, Brisbane, or Adelaide.About the Role:Design and review enterprise-level system architectures.Collaborate with architects, Product Owners, and development teams to shape solution direction.Translate business requirements...

  • Chief Data Platform Architect

    8 hours ago

    Melbourne, Victoria, Australia beBeeDataPlatformManager Full time $139,146 - $164,602

    Establishing a cutting-edge data platform is crucial for organisations seeking to harness the power of data-driven decision-making. Our ideal candidate will lead the delivery of a modern, scalable enterprise-wide data platform to meet the analytical and operational needs of our organisation.About the RoleWe are seeking a seasoned professional with technical...

Chief Data Pipeline Architect

2 weeks ago

Melbourne, Victoria, Australia beBeeCloudDataEngineer Full time $120,000 - $180,000
Job Overview

As a key player in our organization's data infrastructure, you will be responsible for designing and implementing scalable, secure, and highly available data pipelines and platforms. This role involves ensuring efficient continuous integration/continuous deployment (CI/CD) processes, infrastructure automation, and monitoring to support the organization's data ecosystem.

The primary focus areas include infrastructure as code (IaC), CI/CD for data pipelines, monitoring & observability, cost optimization, and cloud security.

To succeed in this position, you will need to have hands-on experience with containerization and orchestration tools, proficiency in infrastructure-as-code (IaC) practices using tools like Terraform, AWS CloudFormation, or Azure Bicep, and expertise in cloud data platforms (AWS, Azure, GCP) with services like AWS Lambda, S3, RDS, Redshift, Glue, EMR, Databricks, BigQuery, etc.

You will also be required to have a strong understanding of cloud networking, security policies, and identity management (IAM, VPC, KMS, Secrets Manager, etc.), as well as experience setting up monitoring and alerting using tools like Prometheus, Grafana, CloudWatch, Datadog, etc.

Additionally, you will need to be able to troubleshoot performance issues and optimize system reliability, and have the ability to write automation scripts for infrastructure provisioning, deployments, and data pipeline management.

As a leader, you will be expected to mentor junior engineers and promote DevOps best practices across teams, and have strong communication skills to work with both technical and non-technical stakeholders.

Responsibilities
  1. Design, implement, and maintain CI/CD pipelines for data engineering workflows.
  2. Automate data pipeline deployments, monitoring, and scaling to ensure reliability and performance.
  3. Develop and enforce infrastructure-as-code (IaC) practices using tools like Terraform, AWS CloudFormation, or Azure Bicep.
  4. Work with Data Engineers and IT teams to optimize ETL/ELT processes and cloud data platforms.
  1. Oversee data infrastructure in cloud environments (AWS, Azure, GCP), ensuring high availability and security.
  2. Implement role-based access control (RBAC), encryption, and network security best practices within data environments.
  3. Collaborate with Security and Compliance teams to ensure regulatory compliance.
  4. Optimize cloud storage, compute, and networking costs while maintaining performance.
  1. Implement monitoring solutions for real-time alerting, anomaly detection, and performance optimization.
  2. Utilize logging and observability tools such as Prometheus, Grafana, Datadog, Splunk, or CloudWatch.
  3. Establish incident response processes, including automated rollback strategies and disaster recovery plans.
  4. Work with Site Reliability Engineering (SRE) teams to ensure high system availability and minimal downtime.
  1. Advocate for DataOps methodologies to improve collaboration between Data Engineering, IT, and Analytics teams.
  2. Implement version control, automated testing, and release management for data pipelines.
  3. Continuously improve CI/CD workflows to enhance deployment frequency and reduce failures.
  4. Enforce best practices for data governance, lineage tracking, and auditability.
  1. Work closely with Data Engineers, Analytics, Security, and IT Operations to streamline data infrastructure.
  2. Act as the subject matter expert (SME) on Data DevOps, providing training and mentorship to team members.
  3. Engage with business leaders and data users to align infrastructure improvements with business goals.
Requirements
  • Bachelor's or Master's degree in Computer Science, Data Engineering, Cloud Computing, or a related field/relevant experience.
  • Experience & Skills:
  • 3-5+ years of experience in DevOps, Data Engineering, or Cloud Infrastructure roles.
  • Hands-on experience with CI/CD tools (e.g., GitHub Actions, Jenkins, Azure DevOps).
  • Proficiency in containerization and orchestration (Docker, Kubernetes, ECS, EKS, AKS) as plus.
  • Experience implementing infrastructure as code (IaC) using Terraform.
  • Expertise in cloud data platforms (AWS, Azure, GCP) with services like AWS Lambda, S3, RDS, Redshift, Glue, EMR, Databricks, BigQuery, etc.
  • Strong understanding of cloud networking, security policies, and identity management (IAM, VPC, KMS, Secrets Manager, etc.).
  • Experience setting up monitoring and alerting using tools like Prometheus, Grafana, CloudWatch, Datadog, etc.
  • Ability to troubleshoot performance issues and optimize system reliability.
  • Strong experience in Python, Bash, or PowerShell for automation and scripting.
  • Ability to write automation scripts for infrastructure provisioning, deployments, and data pipeline management.
  • Ability to lead a small team and mentor junior engineers and promote DevOps best practices across teams.
  • Strong communication skills to work with both technical and non-technical stakeholders.