Big Data Lead Engineer

1 week ago

Sydney, New South Wales, Australia CareCone Group Full time $150,000 - $200,000 per year

Role: Big Data Lead Engineer

Location: Sydney

Job Description:

Technical and Team Leadership:

  • Provides technical direction and guidance to the engineering team.
  • Ensures adherence to engineering standards and best practices.
  • Participates in technical discussions and decision-making.
  • Identifies and resolves technical challenges and issues.
  • Conducts code reviews and provides feedback to other engineers.
  • Maintain high development standards and practices.
  • Facilitates team communication and collaboration.
  • Mentoring and guiding junior engineers.

Technical Skills

  • Proficient with Java, Scala/Spark, API and Streaming – Primary
  • Proficient with Pyspark, Python and SQL.
  • Experience in Azure cloud platform.
  • Expertise in various connectors and pipelines for batch and real-time data collection/delivery.
  • Performance tuning - table partitioning and indexing, process threading.

Process:

  • Should have worked in Agile
  • Experience of working in JIRA and Confluence

Interested people can share their resume at or can reach me on

  • Big Data Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Pyramid Global Technologies Full time $100,000 - $150,000 per year

    Job Description: · years experience working in Data Engineering and Warehousing.· years' experience integrating data into analytical platforms· Experience in ingestion technologies (e.g. Swoop, flume), processing technologies (Spark/Scala), and storage (e.g. HDFS, HBase, Hive)· Experience in data profiling, source-target mappings, ETL development, SQL...

  • Big Data Engineer

    1 week ago

    Sydney, New South Wales, Australia Pyramid Global Technologies Full time $100,000 - $150,000 per year

    Job Description:· years experience working in Data Engineering and Warehousing.· years' experience integrating data into analytical platforms· Experience in ingestion technologies (e.g. Sqoop, flume), processing technologies (Spark/Scala), and storage (e.g. HDFS, HBase, Hive)· Experience in data profiling, source-target mappings, ETL development, SQL...

  • Staff / Lead Data Engineer (AWS, Big Data)

    13 hours ago

    Sydney, New South Wales, Australia Commonwealth Bank Full time $120,000 - $180,000 per year

    Staff Data Engineer (AWS, Big Data)You are determined to stay ahead of the latest Cloud, Big Data and Data warehouse technologies.We're one of the largest and most advanced Data Engineering teams in the country.Together we can build state-of-the-art data solutions that power seamless experiences for millions of customers.Do work that matters:As a Staff Data...

  • Big Data Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Pyramid Global Technologies Full time $80,000 - $120,000 per year

    Job Description for Big Data Engineer:Primary Skills- Bigdata Hadoop, Hive, Spark, scala, SQLSecondary Skills- DW/ETL, Unix, Git, Bitbucket, ControlM- 3-5 years of experience in developing end to end bigdata pipeline- Must have experience in developing at least 1 bigdata project using hive, spark, scala- Thorough understanding of Hadoop and spark...

  • Big Data Lead Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Tata Consultancy Services Full time $180,000 - $220,000 per year

    About the jobAbout TCS:Join Tata Consultancy Services, Asia Pacific and be part of an organization committed to sustainable development for our future. TCS follows the Tata group philosophy of building sustainable businesses that are rooted in the community and demonstrate care for the environment. Our unique values position us to combine a purpose-driven...

  • Lead Data Engineer

    2 weeks ago

    Sydney, New South Wales, Australia INGRITY Full time $120,000 - $180,000 per year

    Location: Sydney Job Type: Contract We are looking for an experienced Lead Data Engineer with deep hands-on expertise in Data and ETL to lead strategic initiatives and transform our data platform. You will leverage your technical leadership, strategic thinking, and problem-solving skills to guide a team of engineers, enhance platform efficiency through AI...

  • Data Engineer

    2 weeks ago

    Sydney, New South Wales, Australia NTT DATA Full time $80,000 - $140,000 per year

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAThe Data Engineer...

  • Lead Data Engineer – AWS

    2 days ago

    Sydney, New South Wales, Australia Bluefin Resources Pty Limited Full time $80,000 - $140,000 per year

    Join a high-performing Data team that's driving a large-scale transformation program within the banking and financial services industry. This is an excellent opportunity for an ambitious and technically strong Data Engineer to play a lead role in shaping a modern, cloud-based data platform. You'll be joining a collaborative, forward-thinking environment...

  • Data Engineer Consultant

    1 week ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time $80,000 - $140,000 per year

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAThe Data Engineer...

  • Lead Data Engineer

    7 hours ago

    Sydney, New South Wales, Australia Macquarie Group Full time $120,000 - $200,000 per year

    Join our high performing Data Team focused on collaborating across a wide range of business and technical stakeholders to deliver a transformation project within Macquarie's Banking and Financial Services division. This is an excellent opportunity for an ambitious, problem solving and technical data professional to contribute and grow rapidly within the...