Data Engineer Professional

3 days ago

Sydney, New South Wales, Australia Optiver Full time
Optiver: A Leader in Global Market Making

As a seasoned Data Engineer, you will play a pivotal role in designing and implementing data solutions that enable efficient access to high-quality data, ensuring smooth operations of our data pipelines. You will work closely with researchers in Sydney and our Global Research and Data Platform teams to productionise workflows, optimise large-scale data processing, and manage infrastructure costs efficiently.

About the Role
  • Collaborate with researchers to transform experimental workflows into production-ready data pipelines that meet performance and reliability standards.
  • Platform and Tooling development: Advancing the start of the art of a variety of Optiver's Research and Data Team's platforms and tools.
  • Cost Optimisation: Monitor and optimise resource usage in partnership with the Data Platform team, balancing performance and cost-effectiveness.
  • Monitoring & Tuning: Implement monitoring tools and fine-tune systems for optimal throughput, latency, and reliability, ensuring seamless access to data.
  • Cross-Functional Collaboration: Work closely with our Data Platform teams globally, maintaining alignment on data strategy, tools, and best practices.
  • Data Pipeline Design & Development: Architect, build, and manage scalable data pipelines using Spark, Databricks, and proprietary HPC tooling, ensuring high availability, scalability, and performance.
  • Documentation & Standards: Develop and maintain clear documentation of data pipeline architectures, processes, and best practices.
  • Mentorship & Guidance: Provide technical mentorship to junior engineers and support the growth of the data engineering function in Sydney.


About You
  • You have 3+ years of experience in data engineering.
  • Demonstrable proficiency in Python.
  • Familiarity with distributed data processing and building robust data pipelines.
  • Familiarity with the use of or concepts underpinning various Data Engineering technologies and approaches.
  • Hands-on experience with Spark or Databricks highly desirable.
  • Ability to lead projects autonomously, prioritise tasks, and deliver high-quality results.
  • Proven success working with research and platform engineering teams to support data-driven projects.
  • Strong problem-solving skills, focusing on efficiency, scalability, and cost-effectiveness.
  • Excellent written and verbal communication skills, with the ability to convey technical concepts to both technical and non-technical stakeholders.
  • A Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related field.
  • Other preferred qualifications include familiarity with cloud-based architectures (e.g. AWS), large-scale relational databases experience (e.g. PostgreSQL), knowledge of real-time data streaming technologies (e.g. Kafka), and familiarity with system languages (e.g. C++, Rust).

  • Data Engineer Professional

    4 days ago

    Sydney, New South Wales, Australia INGRITY Full time

    About the Role: We are seeking an experienced Data Engineer to join our team at INGRITY. As a key member of our data engineering department, you will be responsible for designing and developing scalable data pipelines using Microsoft Fabric tools.Main Responsibilities:Create and manage complex data integration workflows using Microsoft Fabric tools such as...

  • Data Engineer Professional

    3 days ago

    Sydney, New South Wales, Australia Cuscal Limited Full time

    Job OverviewCuscal Limited is a forward-thinking company in the payments sector, pioneering innovative solutions for Australia's financial services industry.We're seeking a skilled Data Engineer to join our team and contribute to the design, development, and support of data products for both internal and client-facing use cases.

  • Data Science and Engineering Professional

    3 days ago

    Sydney, New South Wales, Australia INGRITY Full time

    Role SummaryWe are seeking a highly skilled Data Engineer with expertise in Azure and Power BI to join our team. As a key member of our data engineering team, you will be responsible for building scalable data pipelines, developing robust analytics solutions, and enabling data-driven decision-making for our valued clients.About INGRITYINGRITY is a Microsoft...

  • Data Engineering Leader

    1 day ago

    Sydney, New South Wales, Australia Databricks Full time

    **About the Role at Databricks**This is a senior leadership opportunity to shape the future of big data and machine learning for leading Fortune 500 organisations. As Data Engineering Leader, you will provide strategic leadership for delivering professional services engagements to high-value Databricks customers.In this role, you will be responsible for...

  • Data Engineering Leader

    5 days ago

    Sydney, New South Wales, Australia Culture Amp Full time

    Culture Amp's Data Intelligence Camp is seeking a Data Platform Specialist to join our Data Platform team. This role involves supporting and improving our Data Platforms through data engineering, ML engineering, and data science.Your responsibilities will include:Designing and building data pipelines that ingest data from different sourcesEnsuring those...

  • Data Engineer

    4 days ago

    Sydney, New South Wales, Australia Cuscal Limited Full time

    This is a Data Engineer role with one of the leading companies in AU right now -- Cuscal Limited -- with an amazing team. They are continuing to grow rapidly. This is the chance to join right as the takes off.More About the Role at Cuscal Limited ## **Job Description** The Data Engineer is responsible for the design, development and support of the Data...

  • Data System Engineer

    24 hours ago

    Sydney, New South Wales, Australia Cognizant Full time

    Data Engineering OpportunityCognizant is seeking a highly skilled Data System Engineer to join our team. In this role, you'll be responsible for designing, implementing, and managing cloud-based database solutions on Azure (optional GCP) using SQL Server and PostgreSQL.The Key Responsibilities:Design, implement, and manage cloud-based database solutions on...

  • Data Engineer

    1 day ago

    Sydney, New South Wales, Australia Cuscal Limited Full time

    This is a Data Engineer role with one of the leading companies in AU right now -- Cuscal Limited -- with an amazing team. They are continuing to grow rapidly. This is the chance to join right as the takes off.More About the Role at Cuscal Limited ## **Job Description** **2x Positions Available: We are looking for a Mid-level Data Engineer & Associate Data...

  • Data Engineer Lead

    2 days ago

    Sydney, New South Wales, Australia Macquarie Bank Limited Full time

    At Macquarie, we're shaping the future of global finance. As a Data Engineer Lead, you'll play a crucial role in delivering transformation projects within our Banking and Financial Services business unit. Collaborate with diverse stakeholders to drive technical innovation and excellence.We empower ambitious professionals like you to shape all kinds of...

  • Data Engineering Lead

    1 day ago

    Sydney, New South Wales, Australia INGRITY Full time

    We are a team of experts in data engineering at INGRITY, working closely with our clients to deliver cutting-edge data solutions on Google Cloud Platform (GCP). As a Senior Data Engineer, you will play a key role in designing, developing, and optimising data pipelines, ensuring efficient data processing and analytics.About INGRITY:We are a Microsoft data and...

  • Data Engineer

    2 days ago

    Sydney, New South Wales, Australia Atlassian Full time

    This is a Data Engineer role with one of the leading companies in AU right now -- Atlassian -- with an amazing team. They are continuing to grow rapidly. This is the chance to join right as the takes off.More About the Role at Atlassian **Working at Atlassian** Atlassians can choose where they work – whether in an office, from home, or a combination of...

  • Big Data Engineer

    24 hours ago

    Sydney, New South Wales, Australia FinXL Full time

    About the Project:FinXL is seeking a skilled Data Engineer to join our team and work on a high-profile project with a prominent banking client in Sydney.Key Responsibilities:Develop and implement efficient data ingestion and transformation pipelines using AWS servicesDesign and deploy scalable cloud infrastructure, APIs, and orchestration platformsWork...

  • Senior Data Engineer

    1 day ago

    Sydney, New South Wales, Australia INGRITY Full time

    About INGRITY:We are a Microsoft data and AI solution partner, working closely with Microsoft to create value for our customers. We collaborate with some of the best ASX-listed companies and many medium-sized businesses, delivering transformative data and AI-driven solutions.At INGRITY, we thrive on innovation, customer advocacy, and our strategic...

  • Data Platform Engineer

    1 day ago

    Sydney, New South Wales, Australia Culture Amp Full time

    Culture Amp is a leading employee experience platform that empowers companies to transform employee engagement, drive performance management, and develop high-performing teams.We are seeking a highly skilled Data Engineer to join our Data Platform team. This team is responsible for supporting and improving Culture Amp's Data Platforms through data...

  • Professional Data Science and Engineering Program

    4 days ago

    Sydney, New South Wales, Australia Macquarie Bank Limited Full time

    Embark on a challenging and rewarding career path with our Technology Graduate Program, designed to equip you with the skills and expertise needed to succeed in a rapidly changing industry.Our program combines hands-on experience, mentorship, and formal education to provide a comprehensive learning experience. As a member of our team, you will work on...

  • Data Engineering Expert

    7 days ago

    Sydney, New South Wales, Australia PayTech Nexus Ltd Full time

    Job DescriptionWe are seeking a highly skilled Senior Data Engineer to join our team at PayTech Nexus Ltd.About the Role:This is a 12-month contract opportunity with an ASAP start date.The role is based in Sydney, with a hybrid work arrangement.You will have the opportunity to work with one of the largest and most advanced Data Engineering teams in the...

  • Data Engineering Leader

    7 days ago

    Sydney, New South Wales, Australia HCLTech - Australia and New Zealand Full time

    HCLTech - Australia and New Zealand is a leading technology company that provides innovative solutions to clients across the globe.As a Data Program Director, you will play a critical role in leading our data engineering strategy and delivery across global projects and products.Your primary responsibility will be to ensure excellence in the end-to-end...

  • Data Engineering Team Lead

    5 days ago

    Sydney, New South Wales, Australia Macquarie Bank Limited Full time

    About the RoleAs a seasoned engineering professional, you will lead our data engineering team in developing cutting-edge solutions for Macquarie Bank Limited.Leveraging your technical expertise and leadership skills, you will empower teams to build and maintain scalable, secure, and high-performance data platforms using cloud-based technologies like GEN AI...

  • Data Engineering Lead

    5 days ago

    Sydney, New South Wales, Australia ClearCompany Full time

    **Job Summary:**We are seeking an experienced Senior Data Engineer to lead our data infrastructure design, development, and maintenance. This role requires a strong understanding of big data technologies, cloud-based platforms, and data modelling principles.Main Responsibilities:Design and develop scalable, reliable, and efficient data pipelines, data...

  • Chief Data Engineering Officer

    2 hours ago

    Sydney, New South Wales, Australia Databricks Inc. Full time

    About the Role:We are seeking a seasoned professional to lead our Field Engineering team in Australia. As a Strategic Accounts Director - Solutions Architect, you will be responsible for building and leading a high-performing team of Pre-Sales Solutions Architects focused on large enterprise and strategic customers in ANZ.Your key objectives will...