Data Pipeline Architect

3 days ago

Melbourne, Victoria, Australia beBeeData Full time $120,000 - $160,000

About our organization:

We support millions of Australians in comparing and purchasing personal finance and household products.

Our company makes Australians' lives easier by saving them time, money, and effort. With over 20 years of experience, we are a one-stop destination for comparison and purchasing across insurance, utilities, and personal finance products.

The ideal candidate has expertise in data pipelines, cloud infrastructure, and observability.

The Role:

Data Pipeline Architect designs, implements, and maintains data pipelines and infrastructure for efficient, scalable, and reliable data workflows.

This role focuses on data pipeline best practices, CI/CD automation, cloud infrastructure, and security.

  • Key Responsibilities:
  • Streamline data pipeline deployment through CI/CD
  • Ensure observability, performance, and security in cloud data environments
  • Automate infrastructure management and data workflow orchestration
  • Drive best practices for data governance and cost optimization

What You'll Do:

Data Pipeline Automation & Orchestration

Design, implement, and maintain CI/CD pipelines for automated data workflow deployment.

Develop infrastructure-as-code (IaC) templates for provisioning cloud-based data environments/resources.

Automate ETL/ELT workflow management using orchestration tools like Apache Airflow, Prefect, or AWS Step Functions.

Cloud Infrastructure & Security

Manage data infrastructure in AWS, Azure, or GCP, ensuring high availability and security.

Implement role-based access control (RBAC), encryption, and data security best practices.

Collaborate with Security teams to ensure compliance with industry regulations.

Monitor cloud costs and optimize resource usage for storage, compute, and networking.

Monitoring, Observability & Incident Response

Develop real-time monitoring dashboards and alerts for data pipeline performance and failures.

Use observability tools like Datadog, Prometheus, Grafana, CloudWatch etc. to track latency, errors, and throughput.

Establish incident response workflows and automated rollback strategies for failures.

Work with Site Reliability Engineers (SRE) and DevOps teams to maintain high system uptime and resilience.

DataOps & DevOps Best Practices

Implement version control, automated testing, and release management for data pipelines.

Promote DataOps methodologies to improve collaboration between Data Engineering, IT, and Business teams.

Ensure proper data governance, metadata tracking, and auditability in data workflows.

Collaboration & Stakeholder Engagement

Work closely with Data Engineers, Data Scientists, and Platform Engineers to improve data workflows.

Act as a DataOps champion, providing guidance on best practices for pipeline reliability and security.

Support business teams by ensuring data availability, accessibility, and lineage tracking.

  • Data Pipeline Architect

    3 days ago

    Melbourne, Victoria, Australia beBeeDataConsultant Full time $150,000 - $180,000

    Senior Data ConsultantAs a seasoned Senior Data Consultant, you will be responsible for developing and delivering high-quality data solutions that drive business value.You will work closely with cross-functional teams to design, build, and maintain scalable and reliable data pipelines that extract, transform, and load (ETL/ELT) data from various sources into...

  • Data Architect

    3 weeks ago

    Melbourne, Victoria, Australia TapTalent Full time

    We're Hiring: Data Architect We are seeking an experienced and innovative Data Architect to design and implement robust data solutions that drive business intelligence and analytics. The ideal candidate will have expertise in data modeling, database design, and enterprise data architecture to support our organization's data-driven decision making. Location:...

  • Data Architect

    3 weeks ago

    Melbourne, Victoria, Australia TapTalent Full time

    We're Hiring: Data Architect We are seeking an experienced and innovative Data Architect to design and implement robust data solutions that drive business intelligence and analytics. The ideal candidate will have expertise in data modeling, database design, and enterprise data architecture to support our organization's data-driven decision making. Location:...

  • Data Pipeline Architect

    2 days ago

    Melbourne, Victoria, Australia beBeeDataEngineer Full time $122,736 - $132,847

    Key Data Engineer RoleThe Australian BioCommons team is seeking a highly skilled data engineer to contribute to the establishment of OMIX3, Australia's first integrated, multi-omic analysis platform.Primary Responsibilities:Pipeline Development: Design and implement production-grade tools for automated data flows, including archiving, cataloguing, and...

  • Data Architect Position

    10 hours ago

    Melbourne, Victoria, Australia beBeeData Full time $180,000 - $250,000

    We are seeking a talented Data Platform Architect to lead the design, development and optimisation of our enterprise data platform.About UsWe are a leading financial services company based in Connecticut with over 27,000 employees worldwide. Our team provides information processing and computer software services and products to the financial markets,...

  • Cloud Data Pipeline Developer

    11 hours ago

    Melbourne, Victoria, Australia beBeeData Full time $122,736 - $132,847

    Data Engineering OpportunityWe are seeking an experienced Data Engineer to join our team. This is a permanent opportunity with a strong focus on cloud-based pipeline development.Design, build, and optimize scalable AWS Glue (PySpark) pipelines for structured and semi-structured data sources using industry-standard practices and tools.Develop clear and...

  • Data Architect Specialist

    4 days ago

    Melbourne, Victoria, Australia beBeeDataArchitect Full time $120,000 - $180,000

    Data Architect SpecialistWe are seeking a highly skilled Data Architect Specialist to design and develop scalable data solutions using the Snowflake cloud data platform.Key Responsibilities:Data Pipeline Development: Develop, implement, and manage efficient data pipelines to transfer and process large datasets.Snowflake Integration with Cloud Platforms:...

  • Chief Data Pipeline Architect

    1 day ago

    Melbourne, Victoria, Australia beBeeDataOperationsEngineer Full time $144,000 - $168,000

    As a Senior Data Operations Engineer, you will play a pivotal role in designing and developing data pipelines and integration frameworks for our enterprise data platform. Your expertise will ensure the seamless operation, support, and maintenance of data products and platform services.This is an exciting opportunity to drive improvements in data and...

  • Chief Data Architect

    6 days ago

    Melbourne, Victoria, Australia beBeeDataExpert Full time $190,000 - $220,000

    We are seeking a seasoned data architect to spearhead scalable data solutions across our organization.Key Responsibilities:Design, build and maintain high-performance data pipelines and ETL processes.Work with large datasets in cloud environments to deliver accurate, reliable data insights.Collaborate with cross-functional teams to support analytical and...

  • Senior Data Architect

    2 days ago

    Melbourne, Victoria, Australia beBeeData Full time $125,000 - $175,000

    We are seeking an experienced Data Architect to lead the design and delivery of modern, scalable, and secure data architectures that enable enterprise-wide data capabilities.This role will guide the engineering community through best practices, uplift technical delivery standards, and play a critical role in influencing strategic data and cloud...