Senior Big Data Specialist

2 weeks ago

Sydney, New South Wales, Australia Attribute Group Full time

About Attribute Group

Attribute Group is a leading organization seeking an experienced Senior Big Data Specialist to join our team.

Salary: $130,000 - $140,000 per annum + Super

We are looking for a highly skilled professional with expertise in the Cloudera Data Platform (CDP) and Spark-based ingestion pipelines. This role is ideal for someone with a strong background in building and optimizing data pipelines and a track record of working with large-scale data infrastructures.

Key Responsibilities:

  • Develop, implement, and manage scalable data systems to ensure efficient extraction, transformation, and loading of data from diverse sources using SQL, Spark, and Cloudera technologies.
  • Create high-performance algorithms, predictive models, and prototypes utilizing Spark and Scala.
  • Handle and deploy interactive Hive queries for data analysis and manipulation.
  • Maintain data privacy and compliance in line with data governance standards.
  • Collaborate with stakeholders, including Executive, Product, Data, and Design teams, to address technical issues and support data infrastructure needs.
  • Work with data scientists to transform data and integrate algorithms and models into automated processes.

Qualifications:

  • Proficient in Spark, Scala, and Hive, with practical experience in Cloudera Data Platform (CDP).
  • Extensive experience in building and optimizing big data pipelines, architectures, and data sets.
  • Strong knowledge of Spark SQL and ANSI SQL.
  • Experience in the banking sector, preferably in Payments or Financial Crime.
  • Skilled in data pipeline and workflow management tools.
  • Excellent project management and organizational skills.
  • Proven ability to support and collaborate with cross-functional teams in a dynamic environment.

  • Data Engineer, Solutions Specialist

    1 week ago

    Sydney, New South Wales, Australia Senior Full time

    Job OverviewWe are seeking a highly skilled Data Engineer to join our team as a Senior Solutions Specialist. This role will focus on designing and building innovative data solutions for GroupM clients using Google Cloud Platform.

  • Senior Data Engineer

    3 weeks ago

    Sydney, New South Wales, Australia Experis Full time

    Senior Data Engineer - Big DataExperis is representing a prestigious financial services organisation in recruiting a Senior Data Engineer - Big Data to lead the design, development, and delivery of future-ready data solutions. As a technical expert, you will work closely with cross-functional teams to provide cutting-edge data solutions while mentoring...

  • Senior Data Engineer: Big Data Platform Expert

    5 days ago

    Sydney, New South Wales, Australia Commonwealth Bank of Australia Full time

    We are seeking an experienced Senior Data Engineer to join our team in the Chief Data Office. This is a critical role that will drive the development and implementation of cutting-edge data operations solutions using ETL tools and big data platforms.As a Senior Data Engineer, you will work closely with internal teams and stakeholders to design, build, and...

  • Senior Data Architect

    2 weeks ago

    Sydney, New South Wales, Australia Whizdom Recruitment Full time

    We are seeking a highly skilled Senior Data Architect to join our team in Sydney. As a key member of the project, you will be responsible for driving strategic initiatives by identifying and analysing business needs.Our ideal candidate has over 10 years of experience in IT consulting or similar fields, with strong skills in data mapping, SQL, and business...

  • Senior Data Architect

    2 weeks ago

    Sydney, New South Wales, Australia Whizdom Recruitment Full time

    Unlock Your Potential as a Senior Data Architect with Whizdom RecruitmentAbout the OpportunityWe are seeking an experienced Senior Data Architect to join our team in Sydney. This is a fantastic opportunity to work on high-profile projects, utilizing your expertise in cloud and big data solutions.Job Summary:The successful candidate will be responsible for...

  • Senior Big Data Architect

    2 weeks ago

    Sydney, New South Wales, Australia Attribute Group Full time

    Job OverviewWe are seeking an experienced Data Engineer to join our team at Attribute Group. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining large-scale data systems using Cloudera Data Platform (CDP) and Spark-based ingestion pipelines.About the RoleThis is an excellent opportunity for a...

  • Big Data Solutions Architect

    6 days ago

    Sydney, New South Wales, Australia Commonwealth Bank of Australia Full time

    About Commonwealth Bank of AustraliaWe are one of Australia's most advanced big data operations teams, responsible for CommBank's key analytics capabilities and world-leading capabilities for analytics, information management and decisioning.Our team works across the Group Data Warehouse, Teradata/mainframe, Big Data Platform, Pega Decisioning, SAS, R,...

  • Senior Data Architect for Property Insights

    1 day ago

    Sydney, New South Wales, Australia Data - AI & Data Science Full time

    Company Overview">We shine a light on all things property at Domain Group, a business that aims to simplify the property journey for all involved. Our expertise and exclusive data drive value across the organisation.About the Role">This Senior Data Architect will play a key role in building scalable ETL pipelines using AWS and Snowflake, and designing data...

  • Senior Data Engineer

    4 weeks ago

    Sydney, New South Wales, Australia Commonwealth Bank of Australia Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our team, responsible for designing, building, and delivering enterprise-wide data ingestion, data integration, and data pipeline solutions using common programming languages such as Scala, Java, or Python in a Big Data and Data Warehouse platform.Key ResponsibilitiesDesign and...

  • Big Data ETL Expert

    7 days ago

    Sydney, New South Wales, Australia Commonwealth Bank of Australia Full time

    Unlocking Big Data PotentialWe're seeking an experienced Senior Data Engineer to join our team and drive the delivery of high-quality data operations solutions using ETL tools.About UsThe Commonwealth Bank of Australia is a leading financial institution, empowering seven million customers with seamless experiences through cutting-edge features. Our Big Data...

  • Data Analyst for Big Data Platform

    1 week ago

    Sydney, New South Wales, Australia Randstad Full time

    About the Company and OpportunityRandstad, a specialist recruitment agency, is partnering with a leading Australian bank headquartered in Sydney to find a talented Data Support Analyst for their big data platform.This exciting opportunity has arisen due to the bank's immense growth in their portfolio, and we are looking for an exceptional candidate to...

  • Data Engineer with Big Data Operations Expertise

    1 day ago

    Sydney, New South Wales, Australia Commonwealth Bank of Australia Full time

    Unlock the Power of Data with UsAt the Commonwealth Bank of Australia, we're committed to delivering exceptional customer experiences through cutting-edge data-driven solutions.We're seeking an experienced Senior Data Engineer to join our team and contribute to the development of innovative ETL tools and big data platforms. As a key member of our data...

  • Data Engineer

    1 day ago

    Sydney, New South Wales, Australia Talent Insights Full time

    Job DescriptionThis is an exciting opportunity for an aspiring Data or Machine Learning Engineer to join the team at Talent Insights as a Data Engineer - Big Data Pipeline Specialist.The role involves building and optimizing data pipelines, solutioning with the team to build proof-of-concepts and deploy solutions, working with Data Scientists on Large...

  • Data Architect for Big Data Solutions

    4 days ago

    Sydney, New South Wales, Australia Pyramid Global Technologies Full time

    About Pyramid Global TechnologiesWe are a cutting-edge technology company seeking an experienced Data Architect for Big Data Solutions to lead our team in developing end-to-end big data pipelines.Estimated Salary: $120,000 - $180,000 per yearAs a Data Architect for Big Data Solutions, you will be responsible for designing and implementing scalable big data...

  • Cloud Data Engineer

    6 days ago

    Sydney, New South Wales, Australia Senior Full time

    Job OverviewWe are seeking a seasoned Cloud Data Engineer to join our team as a Marketing Solutions Specialist. As a key member of our data engineering group, you will be responsible for designing and building innovative marketing analytics solutions on Google Cloud Platform (GCP).This role requires a strong background in cloud-based data engineering, with a...

  • Senior Data Engineer for Marketing Analytics and Cloud Solutions

    2 weeks ago

    Sydney, New South Wales, Australia Senior Full time

    About AccelerationAcceleration is a specialist practice for data, analytics & technology consulting. We are a team of highly skilled data and technology practitioners who are inspired by experimentation and new ideas. Our future-forward solutions empower our clients to compete every day in a dynamic environment.We help clients leverage the power of data,...

  • Senior Data Engineer for Cloud Solutions

    1 week ago

    Sydney, New South Wales, Australia Senior Full time

    Job SummaryWe are seeking a highly skilled Senior Data Engineer to join our team in designing and building cloud-based data solutions. As a key member of our engineering team, you will be responsible for delivering innovative data pipelines, data warehouses, and machine learning solutions on Google Cloud Platform.About the RoleThe ideal candidate is a...

  • Lead Data Architect, Big Data Expert

    4 days ago

    Sydney, New South Wales, Australia Commonwealth Bank of Australia Full time

    We are seeking an experienced Senior Data Engineer to join our team at the Commonwealth Bank of Australia.About the RoleThis is a challenging and rewarding opportunity for a seasoned professional with strong expertise in data engineering, architecture, and design.Key ResponsibilitiesDesign, develop, and implement large-scale data warehousing and big data...

  • Data Engineer

    6 days ago

    Sydney, New South Wales, Australia Data - AI & Data Science Full time

    About UsWe shine a light on the property market, simplifying the journey for all involved. Our expertise and exclusive data drive change. At Domain Group, our data department leads the way in driving value across the organisation.The RoleAs a Senior Data Engineer, you'll play a key role in building data pipelines that power Australia's property market....

  • Big Data Solution Architect

    6 days ago

    Sydney, New South Wales, Australia Zone IT Solutions Full time

    Job Title: Big Data Solution ArchitectAbout UsZone IT Solutions is a leading recruitment company based in Australia, specializing in digital, ERP, and larger IT services. We provide flexible, efficient, and collaborative solutions to organizations seeking top-notch IT expertise.Salary RangeThe estimated salary for this position is between $120,000 - $180,000...