Senior Data Engineer

9 hours ago

Sydney, New South Wales, Australia Long Resources Ltd Full time $120,000 - $180,000 per year

Our client an Australian software company known for creating collaboration and productivity tools for teams is looking for a senior software engineer specialising in optimising large-scale data processing systems (batch and streaming) and moving billions of billing records daily. Led by the FinOps Principal Tech Lead, you will collaborate with two other senior/principal engineers on a co-innovation project with AWS.

This 3-month initial contract aiming to reduce latency of AWS cost and usage data by X%, improving our clients ability to manage cloud spending effectively, demonstrated by successful PoC with key use cases including rapid cost anomaly detection, near real-time budget tracking and forecasting.

In a typical day, you will:

  • take ownership of problems from end to end: extracting and cleaning data, understanding source systems, and being involved in driving efficiency in data pipelines and the transformation from data to relevant insights for end users.
  • enhance data quality by adding sources, coding rules, and producing metrics is essential as requirements evolve.
  • do tuning to keep queries performance and improve existing data models to ensure its extensible and well-understood.
  • spar with fellow engineers on solution design and implementation details.
  • help with code reviewing and solution testing

We are looking for an open-minded, structured thinker who is passionate about building services/pipelines that scale. Agility and smart risk-taking are vital in this industry, where digital innovation meets partner and customer needs over time.

On your first day, we'll expect you to have:

  • BS in Computer Science or equivalent experience with 5+ years as Data Engineer or Software developer
  • Strong programming skills in Python and SQL with ability to write performant production-quality code
  • Experience building and operating large scale distributed systems using Amazon Web Services (S3, Kinesis, CloudFormation, AWS Glue, and SNS).
  • Experienced in building production-level and large-scale data engineering pipelines (using Airflow, DBT, Apache Spark, Flink, Hive, and Kafka, Databricks).
  • Design data models for storage and retrieval to meet platform scale and optimisation requirements
  • Agile development mindset, appreciating the benefit of constant iteration and improvement
  • Familiar with modern software development practices (Agile, TDD, CICD) applied to data engineering
  • Enhance data quality through internal tools/frameworks detecting DQ issues. Working knowledge of relational databases and SQL query authoring
  • Excellent communication and collaboration skills
  • Knowledge of FinOps and its modern practices in managing cloud spend

It's great, but not required, if you have

  • Excelling in solving ambiguous and complex problems, being able to navigate through uncertain situations, breaking down complex challenges into manageable components and developing innovative solutions
  • Basic understanding of Machine Learning projects lifecycle

  • Data Engineer

    3 hours ago

    Sydney, New South Wales, Australia NTT DATA Full time $80,000 - $140,000 per year

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAThe Data Engineer...

  • Senior Data Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Maltem Australia Full time $104,000 - $130,878 per year

    Job Title:Senior Data EngineerLocation:Sydney, New South Wales, AustraliaEngagement:Permanent, initial 8-week full-time engagement with potential extensionStart Date:OctoberAbout the Role:Maltem is a global consulting group specialising in digital transformation and data-driven solutions. We partner with leading organisations to deliver impactful results...

  • Senior Software Engineer

    3 hours ago

    Sydney, New South Wales, Australia Seer Data & Analytics Full time $120,000 - $180,000 per year

    Seer Data & Analytics is an Australian purpose-led technology company turning data and AI into action for a better world. Our mission is totransparently and securely connect and​provide access to data from the source to​empower leaders to make better decisions.​​Our values areFairness firstRespect and protectShared groundPioneering spiritTrust by...

  • Senior Data Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Talenza Full time $90,000 - $105,000 per year

    Role: Senior Data EngineerDay Rate: $1150 - $1250 incl. SuperContract Length: 6-monthsLocation: Sydney CBD/WFH FlexWe're looking for a Senior Data Engineer to join a large-scale digital transformation program. You'll play a pivotal role in designing, building, and optimising data pipelines and platform components across a modern cloud-based ecosystem,...

  • Senior Data Engineer

    4 days ago

    Sydney, New South Wales, Australia iVEGA Consulting Full time $100,000 - $180,000 per year

    We have multiple data engineer requirements: Senior Data Engineer, Staff Data Engineer, Lead Data EngineerJD 1: Senior Data Eng position here is the skills summary (the AWS data stack is must have)Sound understanding of Data modelling conceptsStrong knowledge of SQLAWS Data Stack: S3,RDS, Redshift,Iceberg and Glue (proficient)Programming languages:...

  • Senior Data Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Limelight People Full time $104,000 - $130,878 per year

    Senior Data Engineer – SnowflakeSydney Location or Melbourne locationFast growing Data Analytics ConsultancyPermanent position with career development and amazing cultureAbout the Role This Dynamic and rapidly expanding business is looking to add a Senior Data Engineer with deep expertise in Snowflake and modern data architectures to join their growing...

  • Senior Data Engineer

    8 hours ago

    Sydney, New South Wales, Australia Neara Full time $120,000 - $180,000 per year

    Imagine preventing wildfires before they start, or ensuring power grids stay resilient during extreme weather. That's the real-world impact we create every day at Neara.We build 3D physics-enabled digital twins of critical infrastructure networks that help utilities and infrastructure companies run scenarios, assess risks, and make smarter decisions that...

  • Senior Data Engineer

    1 week ago

    Sydney, New South Wales, Australia Pathway Search Full time $100,000 - $175,000 per year

    Senior Data Engineer (Snowflake)Salary: up to $175k + Super Sydney - Remote options available  Help shape the future of retail media by building scalable, high-performance data solutions that power innovation and growth. Key Responsibilities Design, develop, and optimize scalable ETL/ELT data pipelines (batch & streaming).Architect and manage robust data...

  • Senior Data Engineer

    2 days ago

    Sydney, New South Wales, Australia Neara Full time $120,000 - $180,000 per year

    Imagine preventing wildfires before they start, or ensuring power grids stay resilient during extreme weather. That's the real-world impact we create every day at Neara.We build 3D physics-enabled digital twins of critical infrastructure networks that help utilities and infrastructure companies run scenarios, assess risks, and make smarter decisions that...

  • Senior Data Engineer

    8 hours ago

    Sydney, New South Wales, Australia Neara Full time $120,000 - $180,000 per year

    Imagine preventing wildfires before they start, or ensuring power grids stay resilient during extreme weather. That's the real-world impact we create every day at Neara. We build 3D physics-enabled digital twins of critical infrastructure networks that help utilities and infrastructure companies run scenarios, assess risks, and make smarter decisions that...