Data Engineer

5 months ago

Sydney, Australia Attribute Group Full time

Data Engineer
Permanent 
$130k-$140k + Super 

Our client is seeking a seasoned Data Engineer with expertise in the Cloudera Data Platform (CDP) and Spark-based ingestion pipelines. This role is ideal for a professional with a strong background in building and optimising data pipelines and a track record of working with large-scale data infrastructures.

Key Responsibilities:

  • Develop, implement, and manage scalable data systems to ensure efficient extraction, transformation, and loading of data from diverse sources using SQL, Spark, and Cloudera technologies.
  • Create high-performance algorithms, predictive models, and prototypes utilizing Spark and Scala.
  • Handle and deploy interactive Hive queries for data analysis and manipulation.
  • Maintain data privacy and compliance in line with data governance standards.
  • Collaborate with stakeholders, including Executive, Product, Data, and Design teams, to address technical issues and support data infrastructure needs.
  • Work with data scientists to transform data and integrate algorithms and models into automated processes.

Qualifications:

  • Proficient in Spark, Scala, and Hive, with practical experience in Cloudera Data Platform (CDP).
  • Extensive experience in building and optimizing big data pipelines, architectures, and data sets.
  • Strong knowledge of Spark SQL and ANSI SQL.
  • Experience in the banking sector, preferably in Payments or Financial Crime.
  • Skilled in data pipeline and workflow management tools.
  • Excellent project management and organizational skills.
  • Proven ability to support and collaborate with cross-functional teams in a dynamic environment.

If you're interested in the role please APPLY NOW or email me on Jmitchell@attributegroup.com.au

  • Data Engineer

    4 weeks ago

    Sydney, New South Wales, Australia Data - AI & Data Science Full time

    Drive Business Value with Data Engineering at Domain GroupWe are seeking a skilled Data Engineer to join our Data Decision Science team in Sydney. As a key member of our data department, you will play a critical role in building scalable data pipelines that power Australia's property market.About the RoleThis is an exciting opportunity to work with...

  • Data Property Solutions Engineer

    4 weeks ago

    Sydney, New South Wales, Australia Data-Property Data Solns Shared Full time

    We are seeking a highly skilled Solutions Data Engineer to join our team at Data-Property Data Solns Shared. As a Senior member of our organization, you will play a critical role in shaping the future of property data solutions.About UsWe shine a light on all things property, aiming to simplify the property journey for all involved. Our business is motivated...

  • Senior Data Engineer- Data Decision Science Group

    4 weeks ago

    Sydney, Australia Data - AI & Data Science Full time

    Who are weWe shine a light on all things property. Our business aims to simplify the property journey for all involved; motivated by expertise and our exclusive data. Changing the way people engage with property requires a team of diverse thinkers.At Domain Group, our data department is at the forefront of driving value across the organisation. We create...

  • Senior Data Engineer

    3 weeks ago

    Sydney, New South Wales, Australia Data-Property Data Solns Shared Full time

    We are seeking a highly skilled Senior Data Engineer to join our team at Data-Property Data Solutions Shared. This is an exciting opportunity for a seasoned professional to lead the development of cutting-edge data solutions and drive business growth.About UsOur company specializes in simplifying the property journey for all stakeholders, leveraging our...

  • Data Engineering Expert

    4 weeks ago

    Sydney, New South Wales, Australia Data - AI & Data Science Full time

    About UsWe shine a light on all things property, simplifying the journey for everyone involved. Our expertise and exclusive data drive value across the organisation.Role OverviewThis is an exciting opportunity to make your mark in data engineering at Domain Group's Data Decision Science team. You'll play a key role in building data pipelines that power...

  • Senior Azure Data Engineer

    8 months ago

    North Sydney, Australia NTT DATA Full time

    NTT DATA is heavily investing globally in providing state of the art analytical solutions to its clients. Our capability has been focused locally, regionally in APAC and globally by forming strategic partnerships including Microsoft Azure, SAP Analytics, Snowflake, Matillion and many more, leveraging the strength of our cloud and network assets via the NTT...

  • Senior Data Engineer- SVP

    1 month ago

    Sydney, Australia Data-Property Data Solns Shared Full time

    Who are weWe shine a light on all things property. Our business aims to simplify the property journey for all involved; motivated by expertise and our exclusive data. Changing the way people engage with property requires a team of diverse thinkers.Our data department is at the forefront of driving value across the organisation. We create reusable data assets...

  • Senior Data Architect for Property Insights

    4 weeks ago

    Sydney, New South Wales, Australia Data - AI & Data Science Full time

    Company Overview">We shine a light on all things property at Domain Group, a business that aims to simplify the property journey for all involved. Our expertise and exclusive data drive value across the organisation.About the Role">This Senior Data Architect will play a key role in building scalable ETL pipelines using AWS and Snowflake, and designing data...

  • Data Analyst

    6 months ago

    Sydney, Australia The Data Foundry Full time

    Company Description We are a customer-obsessed, “one stop shop” for all things data-related. We are deliberately narrow in our focus areas but we go deep in each of them. We provide solutions, services and training in the **data strategy, data governance, data security, data management, data classification, data wrangling, data lake, data analytics,...

  • Data Engineer

    4 weeks ago

    Sydney, New South Wales, Australia Data-Property Data Solns Shared Full time

    About UsData-Property Data Solns Shared is a leading company that simplifies the property journey for all involved. Our mission is to empower every department with reusable data assets, driving value across the organization.

  • Technical Solutions Engineer

    2 months ago

    Sydney, Australia Cdc Data Centres Full time

    About us: Established in 2007, CDC Data Centres is Australia’s leading operator of sovereign, secure, world-class data centre facilities. Headquartered in Canberra with multiple facilities in the ACT, NSW and New Zealand, CDC’s diversified operations provide secure data centre configurations that support co-location, containerised and hyperscale...

  • Technical Solutions Engineer

    2 months ago

    Sydney, Australia CDC Data Centres Full time

    About us: Established in 2007, CDC Data Centres is Australia’s leading operator of sovereign, secure, world-class data centre facilities. Headquartered in Canberra with multiple facilities in the ACT, NSW and New Zealand, CDC’s diversified operations provide secure data centre configurations that support co-location, containerised and hyperscale...

  • Data Engineer

    2 months ago

    Sydney, Australia Lifebyte Systems Full time

    At LifeByte, we are a dynamic and innovative collective of tech visionaries driven by a relentless pursuit of excellence. Each of us brings a unique set of skills to the table, collaborating on projects that shape the future.Founded in 2017, we are dedicated to fostering an ecosystem of seamless resource exchange, where efficiency and precision are...

  • Data Engineer

    1 month ago

    Sydney, Australia Atlassian Full time

    Working at AtlassianAtlassians can choose where they work – whether in an office, from home, or a combination of the two. That way, Atlassians have more control over supporting their family, personal goals, and other priorities. We can hire people in any country where we have a legal entity. Interviews and onboarding are conducted virtually, a part of...

  • Cybersecurity Expert for NTT DATA

    4 weeks ago

    Sydney, New South Wales, Australia NTT DATA Full time

    About the RoleNTT DATA is seeking an experienced Senior Security Architect to join our team in Australia and New Zealand. This role offers a unique opportunity to work with leading-edge security solutions and make a significant impact on our clients' success.Job DescriptionWe are looking for a skilled Security Architect to provide technical leadership and...

  • Data Engineer

    4 months ago

    Sydney, Australia Mane Consulting Full time

    Are you a customer-obsessed data engineer who can focus on the delivery and innovation of new services? Can you use your Snowflake experience and knowledge to advise, design, and implement cost-effective solutions to our customers, whilst maintaining an excellent delivery posture? Do you want to shape your world and change the way the world works? Do you...

  • Principal Data Engineer

    1 month ago

    Sydney, Australia Talent Insights Full time

    Microsoft Azure, Databricks, Data Factory and Gen AI applications Hybrid, flexible work Opportunity to lead and shape a DE function, greenfield project Principal / Lead Data Engineer ( Microsoft Azure /...

  • Data Engineer

    1 month ago

    Sydney, Australia Experis Full time

    We are seeking an experienced Data Engineer with a strong background in AWS toolsets to advise, develop, and maintain data engineering solutions within the AWS Cloud ecosystem. This role involves designing, building, and managing both batch and real-time data pipelines using various AWS services.Key Responsibilities:Design, build, and operate batch and...

  • Cyber Security Engineer with Expertise in Advanced Threat Detection

    3 weeks ago

    Sydney, New South Wales, Australia NTT DATA Full time

    NTT DATA is a leading provider of digital and AI infrastructure, serving 75% of the Fortune Global 100. We are committed to helping clients innovate, optimize, and transform for long-term success.As a Cyber Security Engineer with NTT DATA, you will play a vital role in securing and resolving cyber threats for our key clients. You will be responsible for...

  • Data Engineer Leader

    4 weeks ago

    Sydney, New South Wales, Australia The Drive Group Full time

    Unlock Your Data Engineering PotentialWe are seeking a talented Data Engineer Leader to join our team at The Drive Group, a fast-growing Australian consultancy. This exciting opportunity will allow you to utilize your expertise in data engineering concepts and techniques to drive business growth.As a Data Engineer Leader, you will be responsible for...