Lead Data Engineer

1 week ago


Sydney, New South Wales, Australia ClearCompany Full time
Lead Data Engineer (Dev Ops)
Sydney, Australia
$220,000 Including Super + Bonus

Overview:


As a Lead Data Engineer reporting to our Engineering Manager, you will be responsible for building, implementing, and enhancing enterprise-scale data platforms.

Working in a Dev Ops environment, you will have end-to-end accountability for designing, developing, deploying, and supporting your data assets.

Additionally, you will create templates, implementation methods, standards, and frameworks.

You will engage with various stakeholders and become an expert in your domain while supporting the broader data engineering and Site Reliability Engineering (SRE) community.

Your role will include continuously evaluating and recommending improvements to infrastructure and processes, leading incident response efforts, and participating in post-mortem analysis.

Key Requirements

Lead significant data engineering projects in an enterprise environment.
Mentor and guide a team on best practices for data engineering and site reliability (SRE).

Utilize Big Data and Hadoop technologies with a focus on Spark, Hive, Presto, and big data storage formats such as Parquet, ORC, Avro.

Manage CI/CD tools like Bamboo/Ansible.
Operate within a cloud-based environment, preferably AWS or GCP.
Demonstrate strong skills in Python, SQL, and shell scripting.
Work with ETL tools such as Talend, Informatica, Datastage, and understand programming languages like Scala.
Apply broad knowledge of technical solutions, design patterns, and coding for complex applications in a distributed computing environment.

If you feel you possess the relevant skills to the above, please respond with most updated CV
#J-18808-Ljbffr

  • Sydney, New South Wales, Australia NTT DATA Full time

    Job Description NTT DATA is heavily investing globally in providing state of the art analytical solutions to its clients. Our capability has been focused locally, regionally in APAC and globally by forming strategic partnerships including Microsoft Azure, SAP Analytics, Snowflake, Matillion and many more, leveraging the strength of our cloud and network...

  • Lead Data Engineer

    3 weeks ago


    Sydney, New South Wales, Australia Randstad Australia Full time

    About the company: My client is a leading global multinational investment banking and financial services company. You will be joining the data engineering team to help build the best technology company in the financial services industry. You will play a leading role in shaping and driving the next evolution of their data and analytics platform, providing...

  • Lead Data Engineer

    1 month ago


    Sydney, New South Wales, Australia Randstad Australia Full time

    About the company: My client is a leading global multinational investment banking and financial services company. You will be joining the data engineering team to help build the best technology company in the financial services industry. You will play a leading role in shaping and driving the next evolution of their data and analytics platform, providing...

  • Lead Data Engineer

    1 week ago


    Sydney, New South Wales, Australia Talenza Full time

    Lead Data Engineer (DevOps) Sydney, Australia $220,000 Including Super + Bonus Overview: As a Lead Data Engineer reporting to our Engineering Manager, you will be responsible for building, implementing, and enhancing enterprise-scale data platforms. Working in a DevOps environment, you will have end-to-end accountability for designing, developing,...

  • Lead Data Engineer

    2 days ago


    Sydney, New South Wales, Australia Randstad Australia Full time

    About the company: My client is a leading global multinational investment banking and financial services company. You will be joining the data engineering team to help build the best technology company in the financial services industry. You will play a leading role in shaping and driving the next evolution of their data and analytics platform, providing...

  • Lead Data Engineer

    1 week ago


    Sydney, New South Wales, Australia Troocoo Full time

    About our client: Our client is an instantly recognisable household name within the Financial Services industry. In recent years they have led the way in the creation of wealth management products and services for their customers. Due to this expansion in their offering and associated customer acquisition, they are seeking a Lead Data Engineer to join and...

  • lead data engineer

    1 week ago


    Sydney, New South Wales, Australia Randstad Full time

    About the company: My client is a leading global multinational investment banking and financial services company. You will be joining the data engineering team to help build the best technology company in the financial services industry. You will play a leading role in shaping and driving the next evolution of their data and analytics platform, providing...

  • Lead Data Engineer

    1 week ago


    Sydney, New South Wales, Australia Macquarie Bank Limited Full time

    Join our Banking and Financial Services data engineering team to help build the best technology company in the financial services industry. Help us build the next evolution of our Data and Analytics Platforms. Joining our team means you'll get to use innovative approaches to data analytics and play a lead role in shaping and driving this forward. At...

  • Lead Data Engineer

    1 week ago


    Sydney, New South Wales, Australia Thoughtworks Full time

    Data ArchitectWe are looking for a skilled professional to join our team as a Data Architect. At our company, you'll have the opportunity to work on cutting-edge data projects and collaborate with talented teammates to solve challenging business problems. Take a look at the responsibilities and requirements below:Develop and implement modern data...

  • Lead Data Engineer

    1 week ago


    Sydney, New South Wales, Australia Referrals Only Full time

    Lead data engineers at Thoughtworks develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. They might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems....

  • Lead Data Engineer

    1 week ago


    Sydney, New South Wales, Australia Ing Bank N.v. Full time

    Great new opportunity for a "Lead Data Engineer – Cloud" to join our Data tribe at ING.Reporting to the Program Director, this role will be working with large volumes of data, both structured and unstructured, and implementing data solutions on the Azure platform.You'll be responsible for design and build of complex data pipelines and extending platform...

  • Data Engineering

    1 week ago


    Sydney, New South Wales, Australia Randstad Full time

    About the company: My client is a leading global multinational investment banking and financial services company. You will be joining the data engineering team to help build the best technology company in the financial services industry. You will play a leading role in shaping and driving the next evolution of their data and analytics platform, providing...

  • Data Engineer

    1 week ago


    Sydney, New South Wales, Australia HiTech Group Full time

    Leading Federal Government Department Initial 12 month contract + 12 month extension Rewarding daily rates $$$ A dynamic Federal Government Department is looking to engage an experienced Data Engineer to be apart of the department's internal key data centric projects. You will be responsible demonstrating strong data engineering and analytics skills with...

  • Data Engineer

    1 week ago


    Sydney, New South Wales, Australia Experis ManpowerGroup Sp. z o.o. Full time

    Our government client are looking for several data engineers to join their team on a contract basis.Please note only Australian Citizens will be considered as security clearance is required.Role Responsibilities and DutiesDesign, construct, and optimize data lakes, CI/CD, and Synapse pipelines within the Azure stack.Develop scalable architecture practices...

  • AWS Data Analyst

    1 week ago


    Sydney, New South Wales, Australia The Data Foundry Full time

    Job DescriptionYour impact As an AWS Data Analyst, you will play a key role in working with customers to understand their business requirements, map these to their data capabilities, liaise with our internal Data Engineers to design and build elegant solutions, then partner with our customers to deploy those solutions in a way that allows them to turn data...


  • Sydney, New South Wales, Australia Data#3 Full time

    Engineering - Software (Information & Communication Technology) Full time Add expected salary to your profile for insights Join Microsoft's largest Australian Partner HRD Employer of Choice and Great Place to Work Brisbane, Sydney or Melbourne Based - Hybrid Working As an Azure Dev Ops Engineer you will work across multiple organisations, managing the latest...

  • Ms Engineer

    1 week ago


    Sydney, New South Wales, Australia Ntt Data Full time

    NTT DATA NTT DATA helps clients transform through consulting, industry solutions, business process services, IT modernization and managed services. View company page Want to be a part of our team?The Security Managed Services Engineer (L4) is responsible for ensuring a managed service is provided to all clients, to ensure that their Security infrastructure...

  • Data Engineer

    1 week ago


    Sydney, New South Wales, Australia Bancroft School Full time

    The Data Engineer is responsible for building and supporting the big data platform.As a Data Engineer, your responsibilities include: Develop, test and support future-ready data solutions for customers across industry verticals Develop, test and support end-to-end batch and near real-time data flows/pipelines Demonstrate understanding in data architectures,...

  • Data Engineers

    1 week ago


    Sydney, New South Wales, Australia CYOS Solutions Full time

    Application closing date: Friday, 14 June 2024 11:59pm, Canberra time Estimated start date: Monday, 01 July 2024 Location of work: NSWWorking arrangements: The team is based in Canberra however, working from home arrangements are considered subject to home-based work application and suitability.Interstate candidates will also be considered.However, there...

  • Data Engineers

    1 week ago


    Sydney, New South Wales, Australia CYOS Solutions Full time

    Application closing date: Friday, 14 June 2024 11:59pm, Canberra timeEstimated start date: Monday, 01 July 2024Location of work:NSWWorking arrangements: The team is based in Canberra however, working from home arrangements are considered subject to home-based work application and suitability.Interstate candidates will also be considered.However, there will...