Senior Data Engineer

1 week ago

Brisbane, Queensland, Australia Compare the Market Full time

Engineering - Software (Information & Communication Technology)

Full time

Add expected salary to your profile for insights

This job ad is the universe's way of telling you that you're currently in the wrong job.

Compare the Market – yes, the one with the meerkats – is a high-growth FinTech company on a transformational journey. We are focused on designing, building, and delivering the very best home loan, energy, insurance, and health comparison experience in Australia. Okay, in the world

We are all about Integrity, Growth, Excellent Customer Service, and being Peerless – without compromise. These values are critical to delivering on our purpose: 'We honestly strive to empower Australians to make the best decisions, saving time, money, and stress.'

About The role:

Are you ready to take your career to new heights? Compare the Market is searching for a Senior Data Engineer to join our innovative team in Toowong. As we continue to evolve our modern data platform on Databricks/AWS, we're dedicated to delivering exceptional value through analytics. Reporting to the Head of Data Science, you'll have the unique opportunity to establish foundational engineering principles and provide technical leadership across business intelligence, data science, and machine learning initiatives.

As a Senior Data Engineer, your expertise extends beyond hands-on development to include guiding architecture and solution design. You'll play a pivotal role in shaping our data strategy from an engineering perspective, contributing to the development of our data platform and ecosystem. Supported by a collaborative team including data scientists, business intelligence developers, data architects, and more, you'll collaborate closely with IT and business teams to drive innovation and excellence.

What You'll Do:

  • Collaborate with data producers, consumers, and support teams to ensure efficient data management for consumption purposes.
  • Design, maintain, and enhance top-tier data pipelines to cater to various analytics use-cases such as business intelligence and machine learning.
  • Partner with architects to devise architectural blueprints for data pipelines, emphasizing event-based real-time processing capabilities.
  • Assist in crafting and executing the data engineering roadmap, aligning it with organisational goals and objectives.
  • Advocate for best practices in data management and governance, encompassing data modelling, ETL efficiency, data quality assurance, observability, and discoverability, as well as security protocols for handling sensitive data and ensuring codebase quality.

What You'll Bring:

  • Extensive experience as a professional data engineer or analytics engineer, specialising in developing ETL pipelines for production environments.
  • Strong background in building analytical data pipelines within cloud systems.
  • Proficiency in data management practices including data modelling, data quality monitoring, maintaining data dictionaries, row/column level security, sensitive data masking or tokenization, and access controls.
  • Ability to write high-quality code for data pipelines, with a preference for Python.
  • Experience with CI/CD and Infrastructure as Code builds.
  • Demonstrated ability to provide data engineering leadership and mentorship.
  • Experience developing on Databricks and/or using Spark or PySpark, building robust feature stores with batch and streaming data for machine learning models, and creating highly automated metadata-driven solutions will be highly regarded.

At Compare the Market, we offer a dynamic and collaborative work environment where your talents will be recognised and rewarded. You'll have the opportunity to make a meaningful impact and contribute to our continued success. As well as ample opportunities for growth and development, we also provide:

  • Vibrant and social community who have annual celebrations, family fun days and regular events.
  • Enjoy flexible work arrangements, including the option of one day working from home each week or a 9-day fortnight.
  • Enjoy additional leave days - 'ME' leave and 'Volunteer Day' leave.
  • Study support (up to 50% of tuition fees for relevant courses) and professional membership reimbursement.
  • Employee discounts on Car, Home, Travel insurance.
  • Income protection insurance provided to support you in the event of non-work-related illness or injury.
  • Discounts with a broad range of providers, including gym memberships, accommodation, dental care and more.
  • Access to the Employee Assistance Program

Compare the Market is an Equal Opportunity Employer, and we encourage all applicants to apply. We acknowledge that we are strengthened by diversity.

Want to know more? Then apply now It's Simples

Please note that this role is only available to candidates with unlimited Australian working rights, as sponsorship is not available.

*Note to Recruiters: We politely ask that you avoid making any approaches or sending any unsolicited resumes to our Recruitment Team or Hiring Leaders across our business.

is a comparison website. What does that mean? Well, pretty much just what the name implies: we provide quotes from a range of participating providers, helping consumers make an informed decision about what is best for them.

The value to you? Comparison websites allow you to get quotes from a variety of sources in one go, rather than having to visit multiple websites and filling in lots of forms.

In very specific terms, what that means is that quickly and conveniently allows consumers to search for some products and services according to their individual needs, and then have a range of quotes from participating providers displayed for them in one place.

Source: This is an extract from the company's own website.

is a comparison website. What does that mean? Well, pretty much just what the name implies: we provide quotes from a range of participating providers, helping consumers make an informed decision about what is best for them.

The value to you? Comparison websites allow you to get quotes from a variety of sources in one go, rather than having to visit multiple websites and filling in lots of forms.

In very specific terms, what that means is that quickly and conveniently allows consumers to search for some products and services according to their individual needs, and then have a range of quotes from participating providers displayed for them in one place.

Source: This is an extract from the company's own website.

Don't provide your bank or credit card details when applying for jobs.

#J-18808-Ljbffr

  • Senior Data Engineer

    1 week ago

    Brisbane, Queensland, Australia Blackroc Full time

    BLACKROC are seeking a Senior Data Engineer for a global technology business who will play a crucial role in designing, building, and optimising data infrastructures, collaborating with senior stakeholders and mentoring team members.You'll be joining a newly formed team charged with owning, operating and improving mission-critical data infrastructure that's...

  • Senior Data Engineer

    3 weeks ago

    Brisbane, Queensland, Australia Randstad Australia Full time

    Randstad Digital are seeking a highly skilled and Senior Data Engineer to play a crucial role in designing, developing, and deploying data pipelines for our client's lake house platformKey Responsibilities & Deliverables:Design and maintain data pipelines for ETL processes.Create data models to meet business needs.Integrate data from various systems for a...

  • Senior Data Engineer

    1 week ago

    Brisbane, Queensland, Australia Randstad Australia Full time

    Randstad Digital are seeking a highly skilled and Senior Data Engineer to play a crucial role in designing, developing, and deploying data pipelines for our client's lake house platformKey Responsibilities & Deliverables:Design and maintain data pipelines for ETL processes.Create data models to meet business needs.Integrate data from various systems for a...

  • Senior Data Engineer

    3 weeks ago

    Brisbane, Queensland, Australia Randstad Australia Full time

    Randstad Digital are seeking a highly skilled and Senior Data Engineer to play a crucial role in designing, developing, and deploying data pipelines for our client's lake house platformKey Responsibilities & Deliverables:Design and maintain data pipelines for ETL processes.Create data models to meet business needs.Integrate data from various systems for a...

  • senior data engineer

    1 week ago

    Brisbane, Queensland, Australia Randstad Full time

    Randstad Digital are seeking a highly skilled and Senior Data Engineer to play a crucial role in designing, developing, and deploying data pipelines for our client's lake house platformKey Responsibilities & Deliverables:Design and maintain data pipelines for ETL processes.Create data models to meet business needs.Integrate data from various systems for a...

  • Senior Data Engineer

    1 week ago

    Brisbane, Queensland, Australia HKT Full time

    This is a full-time position with hybrid working arrangement (1-2 days in office) Role Overview We are passionate about harnessing the power of data to drive innovation and empower businesses. We are seeking a highly technical and experienced Senior Data Engineer to join our dynamic team. The successful candidate will be a key player in designing,...

  • Senior Data Engineer

    1 week ago

    Brisbane, Queensland, Australia HTK Full time

    This is a Full-time position with a Hybrid working model.About Console ConnectConsole Connect by PCCW Global focuses on simplifying business connectivity to the cloud, offering dedicated and secure connections to cloud service providers worldwide. It is the pioneer global software-defined interconnection platform, making business connectivity more accessible...

  • Senior Data Engineer

    1 week ago

    Brisbane, Queensland, Australia HUMANISED GROUP Full time

    Are you a skilled Data Engineer/Developer looking for an exciting opportunity in the energy sector? Join our client's dynamic project team and be part of a transformation with significant impact12 month contract position.Located right here in Brisbane's CBD.Hybrid working arrangement.As the sole technical member in a team of 3-4, you'll collaborate closely...

  • Senior Data Engineer

    1 week ago

    Brisbane, Queensland, Australia BAE Systems Full time

    Job Details Here's how the job details align with your profile. Job Type Permanent Full-time Location Australia Full Job Description At BAE Systems Australia Do meaningful work that MATTERS ENJOY what you do GROW your career Be YOURSELF We operate in a thriving sector that leads us to exciting and valuable work. Each day, we innovate to...

  • Senior Data Engineer

    1 week ago

    Brisbane, Queensland, Australia HUMANISED GROUP Full time

    Are you a skilled Data Engineer/Developer looking for an exciting opportunity in the energy sector? Join our client's dynamic project team and be part of a transformation with significant impact12 month contract position.Located right here in Brisbane's CBD.Hybrid working arrangement.As the sole technical member in a team of 3-4, you'll collaborate closely...

  • Senior Data Engineer

    1 week ago

    Brisbane, Queensland, Australia Humanised Group Full time

    Are you a skilled Data Engineer/Developer searching for an exciting opportunity in the energy sector? Join our client's dynamic project team and be part of a transformation with significant impact.12-month contract position.Excellent day rate.Located in Brisbane's CBD.Hybrid working arrangement.As the sole technical member in a team of 3-4, you'll...

  • Senior Data Engineer

    1 week ago

    Brisbane, Queensland, Australia Randstad Full time

    Randstad Digital are seeking a highly skilled and Senior Data Engineer to play a crucial role in designing, developing, and deploying data pipelines for our client's lake house platformKey Responsibilities & Deliverables:Design and maintain data pipelines for ETL processes.Create data models to meet business needs.Integrate data from various systems for a...

  • Senior Data Engineer

    7 days ago

    Brisbane, Queensland, Australia Suncorp Group Full time

    Drive the technical vision for implementing new cloud data platform on Databricks to support Data Warehousing, Data Science and Gen AI use cases Experience implementing Databricks along with defining patterns and frameworks Empower and guide a high performing, collaborative team We're never just satisfied with how things are – because we know how...

  • Senior Data Migration Engineer

    1 week ago

    Brisbane, Queensland, Australia KPMG Full time

    Our Enterprise Technology and System Integration team brings together skills in intelligent automation, data engineering and software integration to help our clients align their technology capabilities with their strategic and operational objectives. We also draw on KPMG's extensive national and global resources, adopting a multidisciplinary approach to...

  • Senior Data Engineer

    1 week ago

    Brisbane, Queensland, Australia Michael Page Full time

    About Our Client This globally recognised organisation is an industry leader, founded on over a century of history. With over 8000 staff globally, they are well on their way to transforming the business into a leading Digital and Data led organisation by 2025. A large part of this is building upon and enhancing their newly created Global Azure Data...

  • Senior Azure Data Engineer

    1 week ago

    Brisbane, Queensland, Australia StudentUniverse Full time

    Search and refine your job...Work Type: - Full-timeRegion: - AustraliaLocation: - QueenslandBrand: - Product and TechnologyCategory: Information & TechnologyWe are in need of a Senior Azure Data Engineer to join our Global Data Engineering team. As part of this role, you will be crucial in designing, developing, and implementing data pipelines for Flight...

  • Data Engineer

    1 week ago

    Brisbane, Queensland, Australia u&u Full time

    Our client is seeking an experienced Data Engineer to join a dynamic Data Team. As a Data Engineer, you will play a critical role in designing and maintaining the data architecture, ensuring data quality and actionable insights. If you have experience in data engineering and a passion for leveraging data to drive business value, we want to hear from you. Our...

  • Data Engineer

    1 week ago

    Brisbane, Queensland, Australia u&u Full time

    Our client is seeking an experienced Data Engineer to join a dynamic Data Team. As a Data Engineer, you will play a critical role in designing and maintaining the data architecture, ensuring data quality and actionable insights. If you have experience in data engineering and a passion for leveraging data to drive business value, we want to hear from you.Our...

  • Senior Data Migration Engineer

    1 week ago

    Brisbane, Queensland, Australia KPMG Full time

    Data Engineer Opportunity at Our CompanyWe are looking for a talented Data Engineer to join our Data Migration team. In this role, you will be pivotal in developing ETL processes on Azure Synapse for seamless data migration from clients' on-premise systems to new SaaS platforms. If you're a problem-solver with strong expertise in ETL and Azure Data Factory /...

  • Data Engineers

    1 week ago

    Brisbane, Queensland, Australia CYOS Solutions Full time

    Application closing date: Friday, 14 June 2024 • 11:59pm, Canberra timeEstimated start date: Monday, 01 July 2024Location of work: QLDWorking arrangements: The team is based in Canberra however, working from home arrangements are considered subject to home-based work application and suitability. Interstate candidates will also be considered. However, there...