Senior Data Engineer

We are proudly owned by Auto & General Insurance Company. The A&G group is made up of trusted and well-known brands such as iSelect, Compare the Market and Budget Direct. In late 2024, iSelect and Compare the Market formed a new Aggregation business within the A&G group and now support millions of Australians to compare and buy their personal finance and household products such as insurance, energy, and loans.

iSelect

At iSelect, we're passionate about making Australians' lives easier by saving them time, money, and effort. With over 20 years' experience, we are a one-stop destination for comparison and purchasing across insurance, utilities, and personal finance products. We compare a wide range of brands and our trained comparison experts help customers to compare, select and save more, putting them in control.

Compare the Market

At CTM, our Noble Purpose is to make a difference in the lives of Australians by simplifying the process of making sound financial decisions. This Noble Purpose is our cultural 'north star' guiding our direction, prioritisation and decision making.

How you fit:

As CTM continues to enhance our modern data platform (Databricks/AWS), we want to ensure we can best service our analytical use-cases and drive value for the business. Reporting to the Head of Data Science, you will have the opportunity to establish foundational engineering principals and provide technical leadership to support our business intelligence, data science, and machine learning practices.

As a Senior Data Engineer, in addition to your strong hands-on development experience, you'll be able to provide guidance around architecture and solution design. You will contribute to the data strategy from an engineering perspective and help the broader team in the development of the data platform and overall ecosystem. You will work within and be supported by a great team that includes data scientists, business intelligence developers, data engineers, data architects, data modellers, and business analysts, in addition to the broader IT and business teams.

What you do:

• Work with data producers, data consumers, and other supporting roles to ensure data is well managed for consumption.
• Build, maintain, and optimize high quality data pipelines for serving analytics use-cases, including business intelligence and machine learning.
• Work with architects to develop architectural patterns for data pipelines, including for event based real-time processing.
• Support the development and implementation of data engineering roadmap.
• Champion best practices for data management and data governance. This includes data modelling; ETL and processing performance; data quality, observability, and discoverability; security and sensitive data handling; and codebase quality.

What you need:

• 5+ years of experience as a professional data engineer or analytics engineer, developing ETL pipelines for production environments.
• Solid experience building analytical data pipelines in cloud systems.
• Solid data management experience covering data modelling, data quality monitoring, maintaining data dictionaries, row/column level security, sensitive data masking or tokenization, and access controls.
• Ability to write high quality code for data pipelines, preferably in Python.
• Excellent SQL skills.
• Experience with CI/CD and Infrastructure as Code builds.
• Experience providing data engineering leadership and mentorship.
• Solid written and verbal communication skills.
• Highly Regarded:
• Experience developing on Databricks and/or using Spark or PySpark.
• Experience building robust feature stores with batch and streaming data for use by machine learning models.
• Experience building highly automated metadata driven solutions.

What's in it for you:

• Career Opportunities
• A competitive salary plus generous uncapped commission scheme
• The opportunity to work with a well-known, household brand and public listed company
• Work with a team of highly energised, motivated and driven professionals
• 3 'recharge' days per year (yes, in addition to your annual leave)
• A free coffee each day, a funky café and a state of the art work space
• An impressive Employee Benefits Programs

If you are someone who fosters team spirit, has resilience and aspires to continue learning and developing their skills to drive commercial outcomes, you will have tremendous success in helping us achieve our progressive growth agendas. High energy levels and a love of having fun are also important to working in this energetic business.