Data Engineer

1 month ago

Sydney, Australia Onset Group Full time

Job Description

  • Sydney or Brisbane (Hybrid 3 days WFO 2 days WFH)
  • Salary: $140k - $160k + super 
  • Social good Data & AI

Ever considered the advantages of working in a company where every key decision-maker, from top executives to the company's founders, brings a wealth of experience in data? In such an environment, the ability to scale rapidly becomes second nature and the data engineering team is the backbone of everything they build.

Having grown from 8 - 100 people in the last few years, this business needs x2 Mid-Senior Data Engineers as they continue to grow. 

“We’re a social good, Data & AI platform that acts as a profit center for our customers. As you can imagine, customer retention and advocacy is high” 

That’s how the CTO introduced the business to us.

From architecting robust data pipelines to optimising performance and scalability, you'll play a pivotal role in shaping the future of their platform.

What you’ll be doing…

  • Create and maintain optimal data pipeline architecture

  • Assemble large, complex data sets that meet functional/ non-functional business requirements

  • Identify, design and implement internal process improvements: automating manual processes, optimising data delivery, re-designing infrastructure for greater scalability

  • Build the infrastructure required for optimal ETL of data from a variety of data sources using SQL, Spark and other AWS 'big data' technologies

  • Build analytics tools that utilise the data pipeline to provide actionable insights

What you’ll need:

  • Bachelor's or Master's degree in Mathematics, Software Engineering, or Computer Engineering, providing a solid foundation in mathematical principles, software development methodologies, and computer architecture.

  • 3-5+ years of python experience for data manipulation, automation, ensuring code readability and efficiency

  • Hands-on experience with AWS services: Glue, Lambda, S3, SNS, Step Functions, CloudWatch, EMR, Batch

  • Proven track record of designing and implementing robust data pipelines capable of handling large volumes of data with scalability and efficiency.

  • Experience with big data technologies such as Hadoop ecosystem (HDFS, MapReduce, HBase), Spark, Hive, and Presto, for processing and analyzing large volumes of structured and unstructured data efficiently.

  • Proficiency in web scraping techniques to extract data from diverse sources across the internet, including websites, APIs, and other online repositories (desirable). 

You will need full working rights in Australia. 

If that sounds like you hit apply or get in touch with Ailbhe at The Onset for a confidential chat - ailbhe@theonset.com.au

  • Senior Azure Data Engineer

    4 days ago

    Sydney, New South Wales, Australia NTT DATA Full time

    Job Description NTT DATA is heavily investing globally in providing state of the art analytical solutions to its clients. Our capability has been focused locally, regionally in APAC and globally by forming strategic partnerships including Microsoft Azure, SAP Analytics, Snowflake, Matillion and many more, leveraging the strength of our cloud and network...

  • Senior Azure Data Engineer

    2 months ago

    North Sydney, Australia NTT DATA Full time

    NTT DATA is heavily investing globally in providing state of the art analytical solutions to its clients. Our capability has been focused locally, regionally in APAC and globally by forming strategic partnerships including Microsoft Azure, SAP Analytics, Snowflake, Matillion and many more, leveraging the strength of our cloud and network assets via the NTT...

  • Senior Azure Data Engineer

    4 days ago

    North Sydney Council, Australia NTT DATA Full time

    NTT DATA is heavily investing globally in providing state of the art analytical solutions to its clients. Our capability has been focused locally, regionally in APAC and globally by forming strategic partnerships including Microsoft Azure, SAP Analytics, Snowflake, Matillion and many more, leveraging the strength of our cloud and network assets via the NTT...

  • AWS Data Analyst

    4 days ago

    Sydney, New South Wales, Australia The Data Foundry Full time

    Job DescriptionYour impact As an AWS Data Analyst, you will play a key role in working with customers to understand their business requirements, map these to their data capabilities, liaise with our internal Data Engineers to design and build elegant solutions, then partner with our customers to deploy those solutions in a way that allows them to turn data...

  • Data Analyst

    2 months ago

    Sydney, Australia The Data Foundry Full time

    Company Description We are a customer-obsessed, “one stop shop” for all things data-related. We are deliberately narrow in our focus areas but we go deep in each of them. We provide solutions, services and training in the **data strategy, data governance, data security, data management, data classification, data wrangling, data lake, data analytics,...

  • Data Analyst

    1 week ago

    Sydney, Australia The Data Foundry Full time

    Company Description We are a customer-obsessed, “one stop shop” for all things data-related. We are deliberately narrow in our focus areas but we go deep in each of them. We provide solutions, services and training in the **data strategy, data governance, data security, data management, data classification, data wrangling, data lake, data analytics,...

  • Data Analyst

    2 months ago

    Sydney, Australia The Data Foundry Full time

    Company Description We are a customer-obsessed, “one stop shop” for all things data-related. We are deliberately narrow in our focus areas but we go deep in each of them. We provide solutions, services and training in the **data strategy, data governance, data security, data management, data classification, data wrangling, data lake, data analytics,...

  • Azure Devops Engineer

    4 days ago

    Sydney, New South Wales, Australia Data#3 Full time

    Engineering - Software (Information & Communication Technology) Full time Add expected salary to your profile for insights Join Microsoft's largest Australian Partner HRD Employer of Choice and Great Place to Work Brisbane, Sydney or Melbourne Based - Hybrid Working As an Azure Dev Ops Engineer you will work across multiple organisations, managing the latest...

  • Data Specialist

    2 months ago

    Sydney, Australia Data Addiction Pty Ltd Full time

    **What you’ll need before applying**: - At least 2 years’ experience in building data solutions, preferably in Microsoft Azure. - Experience in delivering projects using Agile/Scrum methods. - Strong problem-solving skills and the ability to deliver value. - Strong verbal and written communication skills. - Tertiary qualifications in Computer Science,...

  • Ms Engineer

    4 days ago

    Sydney, New South Wales, Australia Ntt Data Full time

    NTT DATA NTT DATA helps clients transform through consulting, industry solutions, business process services, IT modernization and managed services. View company page Want to be a part of our team?The Security Managed Services Engineer (L4) is responsible for ensuring a managed service is provided to all clients, to ensure that their Security infrastructure...

  • Data Engineer

    4 days ago

    Sydney, New South Wales, Australia HiTech Group Full time

    Leading Federal Government Department Initial 12 month contract + 12 month extension Rewarding daily rates $$$ A dynamic Federal Government Department is looking to engage an experienced Data Engineer to be apart of the department's internal key data centric projects. You will be responsible demonstrating strong data engineering and analytics skills with...

  • AWS Data Analyst

    3 weeks ago

    Sydney, Australia The Data Foundry Full time

    Job DescriptionYour impact As an AWS Data Analyst, you will play a key role in working with customers to understand their business requirements, map these to their data capabilities, liaise with our internal Data Engineers to design and build elegant solutions, then partner with our customers to deploy those solutions in a way that allows them to turn data...

  • Data Engineer

    4 weeks ago

    Sydney, Australia Woolworths Group Limited Full time

    Data Engineer Permanent role Hybrid and flexible working environment Sydney (Norwest) office location About Woolworths Woolworths Group is a food and everyday needs retailer united by the shared purpose of creating better experiences together for a better tomorrow. With more than 1,400 stores across its Woolworths Supermarkets, Countdown...

  • Data Engineer

    2 months ago

    Sydney, Australia TMGM Full time

    Join our team of talented data engineers and take on the exciting challenge of designing, constructing, installing, testing, and maintaining highly scalable data management systems. Not only will you ensure our systems meet business requirements and adhere to industry practices, but you will also have the opportunity to integrate cutting-edge data management...

  • Data Engineer

    4 days ago

    Sydney, New South Wales, Australia HiTech Group Full time

    Exciting opportunity within a Federal Government Department Initial 12 month contract with EXTENSION options Rewarding hourly rates $$$ One of Australia's principal Federal Government departments is looking to engage a skilled Data Engineer & Data Modeller to assist in collaborating with key business areas in the development and implementation of consumable...

  • Data Engineer

    4 days ago

    Sydney, New South Wales, Australia Experis ManpowerGroup Sp. z o.o. Full time

    Our government client are looking for several data engineers to join their team on a contract basis.Please note only Australian Citizens will be considered as security clearance is required.Role Responsibilities and DutiesDesign, construct, and optimize data lakes, CI/CD, and Synapse pipelines within the Azure stack.Develop scalable architecture practices...

  • Data Engineer

    3 weeks ago

    Sydney, Australia Experis Full time

    Our government client are looking for several data engineers to join their team on a contract basis.Please note only Australian Citizens will be considered as security clearance is required.Role Responsibilities and DutiesDesign, construct, and optimize data lakes, CI/CD, and Synapse pipelines within the Azure stack.Develop scalable architecture practices...

  • Data Engineer

    4 days ago

    Sydney, New South Wales, Australia Atlassian Full time

    We are looking for a Data Engineer to join our UIP - Frontend Observability team.The Frontend Observability team is part of the UI Platform organisation.Our team's mission is to help teams build fast experiences and make sure they stay fast. We build tools for measuring, monitoring and catching regressions in frontend performance.UI Platform delivers...

  • Data Engineering

    4 days ago

    Sydney, New South Wales, Australia Randstad Full time

    About the company: My client is a leading global multinational investment banking and financial services company. You will be joining the data engineering team to help build the best technology company in the financial services industry. You will play a leading role in shaping and driving the next evolution of their data and analytics platform, providing...

  • data engineer

    4 days ago

    Sydney, New South Wales, Australia Opus Recruitment Solutions Full time

    ApplyData Engineer Sydney, Australia Permanent$ $130000 per annum Are you a skilled Data Engineer looking for your next challenge? I am currently seeking a Senior Data Engineer with expertise in Azure, Azure Data Factory, Databricks, Synapse, Power BI, and strong experience in SQL to join my clients dynamic team.Responsibilities:Designing, building, and...