Snr Data Engineer

Snr Data Engineer (Data Modeling, AWS Cloud) – 12 Months Fixed Term ContractYou are passionate about modernising the Human Resources function through strategic and secure management of cloud data products, pipelines and databasesWe are undergoing one of Australia's largest digital transformations and establishing an Engineering function within Human Resources.Together we bring an engineering culture and passion for DevSecOps and GitOps within the cloudDo work that mattersAs a Snr Data Engineer you will be designing, creating and managing the cloud databases and data pipelines that underpin our decoupled cloud architecture and API first approach.
You have proven expertise in database design, data ingestion, transformation, data writing, scheduling and query management within a cloud environment.You will have proven experience and expertise in working with AWS Cloud Infrastructure Engineers, Software/API Developers and Architects to design, develop, deploy and operate data services and solutions that underpin a cloud ecosystem.
You will take ownership and accountability of functional and non-functional design and work within a team of Engineers to create innovative solutions that unlock value and modernise technology designs.You will role model continuous improvement mindset in the team, and in your project interactions, by taking technical ownership of key assets, including roadmaps and technical direction of data services running on our AWS environments.See yourself in our teamThe HR Data Engineering centre of excellence works in an Agile methodology with our Human Resources business to plan, prioritise and deliver simpler, safer and better data products needed to support our business.You will work within the People Analytics team, which are working on multiple Gen AI and ML use cases, building out the data assets that will support our goals of leading in technology and AI.To achieve our objectives, you will use you deep understanding of data modelling and data quality and your extensive experience with SQL to access relational databases such as MS Sql Server and Postgres to identify, transform and validate data required for advanced analytics, machine learning and generative AI requirements.You will use your experience in designing and building reliable and efficient data pipelines preferably using modern cloud services on AWS such as S3, Lambda, Redshift, Glue, etc to process large volumes of data efficiently.Experience with data centric frameworks such as Spark with programming knowledge in Scala or Python is highly advantageous.
As is experience working on Linux with shell and automation frameworks to manage code and infrastructure in a well-structured and reliable manner.Experience with Workday or payroll systems as a source for data integration is also highly regarded.We're interested in hearing from people who:Can design and implement databases for data integration in the enterpriseCan collaborate with business and technical teams in order to design and build critical databases and data pipelinesCan performance tune applications from a database code and design perspectiveCan automate data ingestion and transformation processes using scheduling tools.Monitor and troubleshoot data pipelines to ensure reliability and performance.Have experience working through performance and scaling through horizontal scaling designs vs database tuningCan advise business owners on strategic database direction and application solution designTech skillsWe use a broad range of tools, languages, and frameworks.
We don't expect you to know them all but having significant experience and exposure with some of these (or equivalents) will set you up for success in this team.Experience with designing and implementing new data models, with field mappings and transformation logic from source systems to domain model.AWS Data products such as AWS Glue and AWS EMRAWS S3 ingestion, transformation and writing to databasesProficiency in programming languages like Python, Scala or Java for developing data ingestion and transformation scripts.Strong knowledge of SQL for writing, optimizing, and debugging queries.Familiarity with database design, indexing, and normalization principles.Understanding of data formats (JSON, CSV, XML) and techniques for converting between them.
Ability to handle data validation, cleaning, and transformation.Proficiency in automation tools and scripting (e.g., bash scripting, cron jobs) for scheduling and monitoring data processes.Experience with version control systems (e.g., Git) for managing code and collaboration.Working with us:Whether you're passionate about customer service, driven by data, or called by creativity, a career with CommBank is for you.Our people bring their diverse backgrounds and unique perspectives to build a respectful, inclusive, and flexible workplace with flexible work locations.
One where we're driven by our values, and supported to share ideas, initiatives, and energy.
One where making a positive impact for customers, communities and each other is part of our every day.Here, you'll thrive.
You'll be supported when faced with challenges and empowered to tackle new opportunities.
We're hiring engineers from across all of Australia and have opened technology hubs in Melbourne and Perth.
We really love working here, and we think you will too.We support our people with the flexibility to balance where work is done with at least half their time each month connecting in office.
We also have many other flexible working options available including changing start and finish times, part-time arrangements and job share to name a few.
Talk to us about how these arrangements might work in the role you're interested in.If this sounds like the role for you then we would love to hear from you.Apply today
If you're already part of the Commonwealth