Lead Data Engineer

6 days ago

Sydney, New South Wales, Australia beBeeBigData Full time $90,000 - $120,000
Job Title: Big Data Lead Engineer

We are seeking an experienced technical professional to lead our big data engineering team. The successful candidate will be responsible for providing technical direction and guidance to the team, ensuring adherence to engineering standards and best practices, and participating in technical discussions and decision-making.

The ideal candidate will have a strong background in big data technologies such as Java, Scala/Spark, API and Streaming, and experience with cloud platforms such as Azure. They will also have expertise in various connectors and pipelines for batch and real-time data collection/delivery, as well as performance tuning skills.

About the Job
  • Provide technical direction and guidance to the engineering team.
  • Ensure adherence to engineering standards and best practices.
  • Participate in technical discussions and decision-making.
  • Identify and resolve technical challenges and issues.

The role requires excellent communication and collaboration skills, with the ability to work effectively in an Agile environment. Experience with JIRA and Confluence is also essential.

This is a senior-level position, reporting directly to the Engineering Manager. We offer a competitive salary and benefits package, including opportunities for career growth and professional development.

Requirements
  • Proficient with Java, Scala/Spark, API and Streaming – Primary
  • Proficient with PySpark, Python and SQL
  • Experience in Azure cloud platform
  • Expertise in various connectors and pipelines for batch and real-time data collection/delivery
  • Performance tuning - table partitioning and indexing, process threading
  • Agile methodology experience
  • JIRA and Confluence experience
Benefits
  • Competitive salary
  • Opportunities for career growth and professional development
  • Collaborative and dynamic work environment

We are committed to diversity and inclusion in the workplace, and we encourage applications from qualified candidates of all backgrounds.

  • Data Engineer

    3 days ago

    Sydney, New South Wales, Australia Data#3 Full time $90,000 - $120,000 per year

    3 month initial contract + potential extensionMix of WFH and Central Sydney officeOverview:We are seeking an experienced Data Engineer to join a large-scale program for an initial 3-month contract. The role will focus on building and optimising data pipelines in an Azure environment, working closely with technical teams to deliver efficient, high-quality...

  • Data Engineer

    7 hours ago

    Sydney, New South Wales, Australia Data#3 Full time

    Overview:We are seeking an experienced Data Engineer to join a large-scale program for an initial 3-month contract. The role will focus on building and optimising data pipelines in an Azure environment, working closely with technical teams to deliver efficient, high-quality data solutions.Responsibilities:Design, build, and maintain ETL pipelines using...

  • Data Engineer

    1 day ago

    Sydney, New South Wales, Australia Data#3 Full time

    Overview:We are seeking an experienced Data Engineer to join a large-scale program for an initial 3-month contract. The role will focus on building and optimising data pipelines in an Azure environment, working closely with technical teams to deliver efficient, high-quality data solutions.Responsibilities:Design, build, and maintain ETL pipelines using...

  • Data Engineer

    2 days ago

    Sydney, New South Wales, Australia Data#3 Full time

    Overview:We are seeking an experienced Data Engineer to join a large-scale program for an initial 3-month contract. The role will focus on building and optimising data pipelines in an Azure environment, working closely with technical teams to deliver efficient, high-quality data solutions.Responsibilities:Design, build, and maintain ETL pipelines using...

  • Data Engineer Consultant

    3 weeks ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time

    Join to apply for the Data Engineer Consultant role at NTT DATA, Inc.3 days ago Be among the first 25 applicantsJoin to apply for the Data Engineer Consultant role at NTT DATA, Inc.Get AI-powered advice on this job and more exclusive features.Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for...

  • Data Engineer Consultant

    4 weeks ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time

    Join to apply for the Data Engineer Consultant role at NTT DATA, Inc.3 days ago Be among the first 25 applicantsJoin to apply for the Data Engineer Consultant role at NTT DATA, Inc.Get AI-powered advice on this job and more exclusive features.Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for...

  • Data Engineer Consultant

    2 weeks ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time

    Join to apply for the Data Engineer Consultant role at NTT DATA, Inc.3 days ago Be among the first 25 applicantsJoin to apply for the Data Engineer Consultant role at NTT DATA, Inc.Get AI-powered advice on this job and more exclusive features.Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for...

  • Data Engineering Lead

    3 days ago

    Sydney, New South Wales, Australia Tribus Full time $125,000 - $175,000 per year

    Data Engineering Lead We're looking for a Data Engineering Lead to lead the build-out of scalable data infrastructure for real-time analytics and reporting. This is a leadership-first role (60%) with continued technical delivery (40%). In This Role, You Will:Lead a growing team of data engineers growing to 6. Build and manage streaming data pipelines...

  • Solar Design Engineer

    1 day ago

    Sydney, New South Wales, Australia Electrical Engineer Full time

    OverviewHarvey Norman Commercial Division specialises in the supply of goods & services to project builders, developers, architects and the mining industry. Our clients depend upon Harvey Norman Commercial to supply the best range, the best service and the best prices.We have an exciting opportunity at our Taren Point office for a motivated and driven Full...

  • Solar Design Engineer

    2 days ago

    Sydney, New South Wales, Australia Electrical Engineer Full time

    OverviewHarvey Norman Commercial Division specialises in the supply of goods & services to project builders, developers, architects and the mining industry. Our clients depend upon Harvey Norman Commercial to supply the best range, the best service and the best prices.We have an exciting opportunity at our Taren Point office for a motivated and driven Full...