Senior Data Engineer

3 weeks ago

Sydney, New South Wales, Australia Commonwealth Bank Full time

Join to apply for the Senior Data Engineer (AWS Cloud, API Integration) role at Commonwealth Bank

We're embarking on an exciting Digital Transformation program and are ready to push the boundaries and deliver engineering best practices to elevate the digital experience of our customers.

Overview

Senior Data Engineer focused on building scalable, secure, and high‑performance data platforms that power real‑time and analytical use cases. The role includes contributing to API services and AI‑enabled features that enhance customer experiences.

Roles & Responsibilities
  • Design & Build AWS Data Platforms
  • Architect, implement, and operate data lakes/lakehouses on S3 with Glue/Athena/Redshift, Iceberg/Hudi/Delta; optimise storage layout, partitioning, compaction, and schema evolution.
  • Build batch and streaming pipelines using Glue/EMR/Lambda/Step Functions, Kafka (good to have); design for idempotency, replay, DLQs, and exactly‑once/at‑least‑once semantics.
  • Productionise Airflow orchestration with robust DAG design, SLA management, retry/backoff, and per‑environment configuration.
  • Implement CI/CD pipelines for data workflows and services using AWS native tools (CodePipeline, CodeBuild) or GitHub Actions/Automations; include automated testing and deployment strategies.
  • Ensure data governance, security, and compliance: lineage, cataloguing, DQ checks, PII protection, and privacy‑by‑design.
  • Expose curated data and real‑time context through API services (REST/GraphQL) with proper security, caching, and versioning.
  • Support AI/ML integration for channel use cases by enabling model endpoints and feature pipelines.
  • Collaborate with cross‑functional teams to ensure solutions meet performance, reliability, and operational excellence standards.
  • Mentor engineers on data engineering best practices, orchestration, and automation.
Skills Required
  • Core Data Engineering & AWS Expertise
  • Strong experience with AWS services: S3, Glue, EMR, Lambda, Step Functions, Redshift, Athena, Lake Formation
  • Proficiency in data modelling, SQL, and building ETL/ELT pipelines for structured and semi‑structured data
  • Familiarity with lakehouse technologies (Iceberg/Hudi/Delta) and metadata management. Orchestration & CI/CD
  • Hands‑on experience with Apache Airflow for workflow orchestration
  • Design and implement CI/CD pipelines using AWS CodePipeline/CodeBuild or GitHub Actions
  • Develop automated testing and deployment strategies
  • Leverage Infrastructure as Code tools such as Terraform and CloudFormation
  • Proficiency in Python and SQL for scripting automation and operational tasks
  • Utilise containerisation technologies (e.g., Docker) and understand Kubernetes concepts for scalable deployments
  • Gain exposure to API design and integration (REST/GraphQL) and API gateways
  • Understand AI/ML workflows, including model deployment, monitoring, and basic MLOps practices
Working with us

We're hiring engineers from across all of Australia and have opened technology hubs in Melbourne and Perth. We support our people with flexible work options, including part‑time arrangements and job share where possible. If you require additional support, please contact HR Direct on 1800 989 696.

Expressions of Interest

We're interested in hearing from people who are passionate about building next‑generation data platforms and data pipeline solutions across the bank, and who enjoy collaborating to advance Data Engineering practices.

Seniority level
  • Mid-Senior level
Employment type
  • Full-time
Job function
  • Information Technology

Advertising End Date: 13/09/2025

Location: Sydney, New South Wales, Australia


#J-18808-Ljbffr

  • Data Engineer

    3 weeks ago

    Sydney, New South Wales, Australia Data#3 Full time

    Overview:We are seeking an experienced Data Engineer to join a large-scale program for an initial 3-month contract. The role will focus on building and optimising data pipelines in an Azure environment, working closely with technical teams to deliver efficient, high-quality data solutions.Responsibilities:Design, build, and maintain ETL pipelines using...

  • Data Engineer

    3 weeks ago

    Sydney, New South Wales, Australia Data#3 Full time

    Overview:We are seeking an experienced Data Engineer to join a large-scale program for an initial 3-month contract. The role will focus on building and optimising data pipelines in an Azure environment, working closely with technical teams to deliver efficient, high-quality data solutions.Responsibilities:Design, build, and maintain ETL pipelines using...

  • Data Engineer

    1 week ago

    Sydney, New South Wales, Australia NTT DATA Full time $80,000 - $140,000 per year

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAThe Data Engineer...

  • Data Engineer

    3 days ago

    Sydney, New South Wales, Australia Data#3 Full time $120,000 - $150,000 per year

    3 month initial contract + potential extensionMix of WFH and Central Sydney officeOverview:We are seeking an experienced Data Engineer to join a large-scale program for an initial 3-month contract. The role will focus on building and optimising data pipelines in an Azure environment, working closely with technical teams to deliver efficient, high-quality...

  • Senior Data Engineer

    4 weeks ago

    Sydney, New South Wales, Australia Equal Experts Full time

    Direct message the job poster from Equal ExpertsAt Equal Experts, we're a network of over 3500 experienced technology practitioners globally, who value autonomy, collaboration, and delivering meaningful outcomes for our clients. Our teams are made up of senior professionals who bring deep expertise across software engineering, data, product, and delivery.We...

  • Senior Data Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Talenza Full time

    Base pay rangeA$1,150.00/daily - A$1,250.00/dailyDirect message the job poster from TalenzaSenior Data Engineer - NSW Public ServicesRole: Senior Data EngineerDay Rate: $1150 - $1250 incl. SuperContract Length: 6-monthsLocation: Sydney CBD/WFH FlexWe're looking for a Senior Data Engineer to join a large-scale digital transformation program. You\'ll play a...

  • Senior Data Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Talenza Full time

    Base pay rangeA$1,150.00/daily - A$1,250.00/dailyDirect message the job poster from TalenzaSenior Data Engineer - NSW Public ServicesRole: Senior Data EngineerDay Rate: $1150 - $1250 incl. SuperContract Length: 6-monthsLocation: Sydney CBD/WFH FlexWe're looking for a Senior Data Engineer to join a large-scale digital transformation program. You\'ll play a...

  • Senior Data Engineer

    4 weeks ago

    Sydney, New South Wales, Australia Blackmores Group Full time

    Join to apply for the Senior Data Engineer role at Blackmores GroupJoin to apply for the Senior Data Engineer role at Blackmores GroupGet AI-powered advice on this job and more exclusive features.As Australia's most trusted manufacturer of natural health products 15 years running, and one of the Financial Reviews Best Places to Work in 2023, it couldn't be a...

  • Data Engineer Consultant

    6 days ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time $80,000 - $140,000 per year

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAThe Data Engineer...

  • Senior Data Engineers

    23 hours ago

    Sydney, New South Wales, Australia Adactin Group Pty. Ltd. Full time $120,000 - $180,000 per year

    About AdactinAdactin Group is a 100% Australian-owned global IT consulting company, headquartered in Sydney. We are a trusted technology partner helping organisations drive end-to-end digital transformation through our deep expertise in Microsoft and AWS ecosystems, AI-powered Digital Engineering, and future-ready Quality Assurance.Our managed delivery...