Transformative Data Engineer

1 week ago

Sydney, New South Wales, Australia beBeeDataEngineer Full time $150,000 - $200,000
Job Overview

As a Senior Data Engineer, you will play a key role in driving innovation and outcomes in data engineering initiatives. You will provide technical leadership, mentor junior engineers, and collaborate with business and product teams while advancing your skills with the latest technologies and industry best practices.

To succeed in this role, you must be a strategic thinker who can translate complex business requirements into technical solutions through effective collaboration, strong communication skills, and adaptable problem-solving approach.

Key Responsibilities:
  • Provide technical leadership in the data space, collaborating with cross-functional teams to ensure successful execution of data initiatives.
  • Act as a key driver of technical delivery, proactively identifying risks and solutions, and coordinating with engineers, BAs, QAs, and project managers to ensure timely delivery.
  • Lead technical solution design and troubleshooting, ensuring the team builds scalable and maintainable systems that meet business needs.
Technical Requirements:
  • Ensure adherence to data privacy, security, observability, and governance standards.
  • Collaborate with architects and senior engineers to design resilient, high-performance data solutions using cloud-based technologies.
  • Build scalable data pipelines to ingest, process, and expose data using AWS services such as Kinesis, Athena, Redshift, DynamoDB, Lambda, and S3.
  • Develop batch and streaming data solutions using Apache Kafka, Apache Spark, Airflow, and Delta Lake for real-time and scheduled data processing.
  • Design and implement event-driven data architectures with streaming frameworks such as Kafka Streams and Kinesis Data Firehose.
  • Implement Data Quality, Data Lineage, and Observability frameworks to ensure data integrity and reliability.
  • Build CI/CD pipelines using tools such as GitHub Actions, Terraform.
  • Collaborate with Site Reliability Engineers to ensure stability and uptime of production pipelines and services.
  • Continuously identify opportunities for performance improvement, cost optimisation, and process efficiency.
  • Work with Data Scientists to ensure high-quality data for Machine Learning models and integrate them into production-ready pipelines.
Requirements:
  • Undergraduate degree focused on Computer Science, Engineering, Data Science or a related discipline.
  • 5+ years of hands-on experience in data profiling, source-to-target mapping, ETL development, SQL optimization, testing, and implementation.
  • Proven experience delivering large-scale data ingestion, transformation, and processing using AWS services.
  • Proficient with modern data platforms and tools including Snowflake, Databricks, and Apache Storm.
  • Strong experience working with relational databases (SQL) as well as NoSQL databases, particularly MongoDB.
  • Proven ability to work effectively in a fast-paced environment and adapt to changing priorities and deadlines.
  • Exposure to CI/CD tools and practices, including Ansible, CloudFormation, Jenkins, and related automation technologies.
  • Experience developing and deploying solutions on cloud platforms such as AWS, GCP, or Azure.

  • Data Engineer

    7 days ago

    Sydney, New South Wales, Australia Data#3 Full time $90,000 - $120,000 per year

    3 month initial contract + potential extensionMix of WFH and Central Sydney officeOverview:We are seeking an experienced Data Engineer to join a large-scale program for an initial 3-month contract. The role will focus on building and optimising data pipelines in an Azure environment, working closely with technical teams to deliver efficient, high-quality...

  • Data Engineer

    4 days ago

    Sydney, New South Wales, Australia Data#3 Full time

    Overview:We are seeking an experienced Data Engineer to join a large-scale program for an initial 3-month contract. The role will focus on building and optimising data pipelines in an Azure environment, working closely with technical teams to deliver efficient, high-quality data solutions.Responsibilities:Design, build, and maintain ETL pipelines using...

  • Data Engineer

    5 days ago

    Sydney, New South Wales, Australia Data#3 Full time

    Overview:We are seeking an experienced Data Engineer to join a large-scale program for an initial 3-month contract. The role will focus on building and optimising data pipelines in an Azure environment, working closely with technical teams to deliver efficient, high-quality data solutions.Responsibilities:Design, build, and maintain ETL pipelines using...

  • Data Engineer

    5 days ago

    Sydney, New South Wales, Australia Data#3 Full time

    Overview:We are seeking an experienced Data Engineer to join a large-scale program for an initial 3-month contract. The role will focus on building and optimising data pipelines in an Azure environment, working closely with technical teams to deliver efficient, high-quality data solutions.Responsibilities:Design, build, and maintain ETL pipelines using...

  • Transformative Data Engineer

    6 days ago

    Sydney, New South Wales, Australia beBeeInformatica Full time $120,000 - $160,000

    Job OverviewTransformative Data Engineer with Expertise in Informatica Required to Lead Scalable ETL Solutions.The formation of the UNITE Data Engineering Team is a key initiative to design and lead robust data transformation solutions. As part of this high-performance team culture, you will have the opportunity to work on a transformative program-one of the...

  • Data Engineer Consultant

    4 weeks ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time

    Join to apply for the Data Engineer Consultant role at NTT DATA, Inc.3 days ago Be among the first 25 applicantsJoin to apply for the Data Engineer Consultant role at NTT DATA, Inc.Get AI-powered advice on this job and more exclusive features.Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for...

  • Data Engineer Consultant

    4 weeks ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time

    Join to apply for the Data Engineer Consultant role at NTT DATA, Inc.3 days ago Be among the first 25 applicantsJoin to apply for the Data Engineer Consultant role at NTT DATA, Inc.Get AI-powered advice on this job and more exclusive features.Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for...

  • Data Engineer Consultant

    2 weeks ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time

    Join to apply for the Data Engineer Consultant role at NTT DATA, Inc.3 days ago Be among the first 25 applicantsJoin to apply for the Data Engineer Consultant role at NTT DATA, Inc.Get AI-powered advice on this job and more exclusive features.Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for...

  • Transformation Data Engineer

    7 days ago

    Sydney, New South Wales, Australia beBeeData Full time $80,000 - $120,000

    Job TitleAn exciting opportunity to transform data into actionable insights exists within our organization. This role involves working collaboratively with a team to design and develop data engineering solutions that cater to advanced analytical use cases.Key Responsibilities:Transform design and product vision into working solutions for end users, fostering...

  • Senior Data Transformation Engineer

    15 minutes ago

    Sydney, New South Wales, Australia beBeeDataTransformation Full time $149,999 - $199,999

    Data Transformation SpecialistWe are embarking on a journey to elevate data quality and availability in our domain through innovative engineering practices.This is an exciting opportunity for a Staff Software Engineer to shape the direction of our data and analytics cloud platforms.Reporting and Insights CrewThis role is part of the Reporting and Insights...