Senior Data Engineer

Overview

Senior Data Engineer (AWS Cloud, API Integration) – Commonwealth Bank of Australia – Sydney, Australia

Commonwealth Bank of Australia Sydney, Australia

Do Work That Matters

We're driving a major transformation in Marketing & Corporate Affairs (MCA) across Customer Service Channels (voice, chat, email, social, web & mobile). This role focuses on building scalable, secure, and high-performance data platforms that power real-time and analytical use cases. You/'ll also contribute to API services and AI-enabled features that enhance customer experiences.

You'll design and deliver data solutions that enable personalised service, operational insights, and intelligent automation across multiple customer touchpoints. Your work will underpin real-time decisioning, reporting, and AI-driven capabilities for MCA.

See Yourself in Our Team

You'll join the Customer Service Channels domain within MCA, partnering with CDAO and channel teams to deliver data pipelines, orchestration frameworks, and integration patterns that support both batch and streaming use cases. While your core strength is data engineering, you/'ll also help enable API exposure and AI/ML integration where needed.

Roles & Responsibilities

- Design & Build AWS Data Platforms
- Architect, implement, and operate data lakes/lakehouse on S3 with Glue/Athena/Redshift, Iceberg/Hudi/Delta; optimise storage layout, partitioning, compaction, and schema evolution.
- Build batch and streaming pipelines using Glue/EMR/Lambda/Step Functions, Kafka (good to have); design for idempotency, replay, DLQs, and exactly-once/at-least-once semantics.
- Productionise Airflow orchestration with robust DAG design, SLA management, retry/backoff, and per-environment configuration.
- Implement CI/CD pipelines for data workflows and services using AWS native tools (CodePipeline, CodeBuild) or GitHub Actions/Automations; include automated testing and deployment strategies.
- Ensure data governance, security, and compliance: lineage, cataloguing, DQ checks, PII protection, and privacy-by-design.
- Expose curated data and real-time context through API services (REST/GraphQL) with proper security, caching, and versioning.
- Support AI/ML integration for channel use cases (e.g., summarisation, classification, RAG pipelines) by enabling model endpoints and feature pipelines.
- Collaborate with cross-functional teams to ensure solutions meet performance, reliability, and operational excellence standards.
- Mentor engineers on data engineering best practices, orchestration, and automation.

What We/'re Looking For

- Are passionate about building next generation data platforms and data pipeline solution across the bank.
- Constantly thinking outside the box and breaking boundaries to solve complex data problems.
- Can collaborate, co-create and contribute to existing Data Engineering practices in the team.

Skills Required

We use a broad range of tools, languages, and frameworks. We don/'t expect you to know them all but experience or exposure with some of these (or equivalents) will set you up for success in this team;

- Core Data Engineering & AWS Expertise
- Strong experience with AWS services: S3, Glue, EMR, Lambda, Step Functions, Redshift, Athena, Lake Formation.
- Proficiency in data modelling, SQL, and building ETL/ELT pipelines for structured and semi-structured data.
- Familiarity with lakehouse technologies (Iceberg/Hudi/Delta) and metadata management.
- Orchestration & CI/CD
- Hands-on experience with Apache Airflow for workflow orchestration.
- Design and implement CI/CD pipelines using AWS CodePipeline/CodeBuild or GitHub Actions
- Develop automated testing and deployment strategies
- Leverage Infrastructure as Code tools such as Terraform and CloudFormation
- Demonstrate proficiency in Python and SQL for scripting automation and operational tasks
- Utilise containerisation technologies (e.g., Docker) and understand Kubernetes concepts for scalable deployments
- Gain exposure to API design and integration (REST/GraphQL) and API gateways
- Understand AI/ML workflows, including model deployment, monitoring, and basic MLOps practices

Working with us

Whether you/'re passionate about customer service, driven by data, or called by creativity, a career with CommBank is for you.

Here, you/'ll thrive. You/'ll be supported when faced with challenges and empowered to tackle new opportunities. We/'re hiring engineers from across all of Australia and have opened technology hubs in Melbourne and Perth. We really love working here, and we think you will too.

We support our people with the flexibility to balance where work is done with at least half their time each month connecting in office. We also have many other flexible working options available including changing start and finish times, part-time arrangements and job share to name a few. Talk to us about how these arrangements might work in the role you/'re interested in.

If this sounds like the role for you then we would love to hear from you. Apply today

If you/'re already part of the Commonwealth Bank Group (including Bankwest, x15ventures), you/'ll need to apply through Sidekick to submit a valid application. We/'re keen to support you with the next step in your career.

We/'re aware of some accessibility issues on this site, particularly for screen reader users. We want to make finding your dream job as easy as possible, so if you require additional support please contact HR Direct on 1800 989 696.

Advertising End Date: 11/10/2025

More jobs From Commonwealth Bank of Australia

#J-18808-Ljbffr