Lead Data Engineer

3 weeks ago

Canberra, ACT, Australia ClearCompany Full time

Lead Data Engineer

Sydney, Australia

$230,000 (including Super) + Bonus

About the Role:

Join a high-performing Data Team at a leading financial services institution to lead regulatory initiatives and drive innovation in data engineering, utilizing AI and automation to enhance platform efficiency.

Key Responsibilities:

  • Lead the development and transformation of the regulatory reporting platform.
  • Collaborate across business and technical teams to deliver complex data solutions.
  • Implement and optimize data integration pipelines and data warehouse solutions.
  • Oversee the design and architecture of large-scale data solutions using AWS services.
  • Drive platform efficiency through automation and AI enhancements.
  • Ensure data governance and compliance with internal and external regulations.
  • Provide leadership and mentorship to data engineers, fostering a culture of innovation and excellence.

Required Skills and Experience:

  • Proficient in Spark, Hive, Presto, and big data storage formats (Parquet, ORC, Avro).
  • Experience with data pipeline orchestration tools such as Airflow.
  • Hands-on experience with data integration tools (Talend, DataStage, or Informatica).
  • Strong AWS experience, including EC2, S3, RDS, and EMR.
  • Expertise in data modeling techniques and data architecture best practices.
  • Knowledge of data lineage, technical metadata, data quality, and reconciliation processes.
  • Advanced skills in Spark SQL and Python for data automation.
  • Ability to tune and optimize data queries for performance.
  • Strong communication and negotiation skills.
  • Proven experience in leading cross-platform data solutions and complex projects.
  • Ability to mentor and lead technical teams.

If you feel you possess the relevant skills to the above, please respond with most updated CV

#J-18808-Ljbffr

  • Azure Data Factory Engineer

    3 weeks ago

    Canberra, ACT, Australia Babylon Data Full time

    Job Title: Azure Data Factory EngineerRate Guide: $100–$125 per hour (plus super @ 12%)Location of Work: Canberra, ACTContract Duration: ASAP – 30 June 2026Start Date: Tuesday, 23 September 2025Maximum Hours per Week: Up to 40 hoursSecurity Clearance Requirements: Must be an Australian Citizen and able to obtain NV1 clearanceJob DetailsBabylon Data is...

  • Azure Data Factory Engineer

    3 weeks ago

    Canberra, ACT, Australia Babylon Data Full time

    Job Title: Azure Data Factory EngineerRate Guide: $100–$125 per hour (plus super @ 12%)Location of Work: Canberra, ACTContract Duration: ASAP – 30 June 2026Start Date: Tuesday, 23 September 2025Maximum Hours per Week: Up to 40 hoursSecurity Clearance Requirements: Must be an Australian Citizen and able to obtain NV1 clearanceJob DetailsBabylon Data is...

  • Lead Data Engineer

    3 weeks ago

    Canberra, ACT, Australia ClearCompany Full time

    Lead Data EngineerSydney, Australia$230,000 (including Super) + BonusAbout the Role:Join a high-performing Data Team at a leading financial services institution to lead regulatory initiatives and drive innovation in data engineering, utilizing AI and automation to enhance platform efficiency.Key Responsibilities:Lead the development and transformation of the...

  • Lead Data Analyst

    3 weeks ago

    Canberra, ACT, Australia NTT DATA Full time

    Lead Data Analyst - Baseline Security ClearanceNTT DATA Canberra, Australian Capital Territory, AustraliaOur client is looking for a seasoned specialist in the Microsoft industry and international customer relations to aid in a contract review and discovery initiative. The ideal candidate will be responsible for identifying and analyzing existing...

  • Lead Data Engineer

    2 weeks ago

    Canberra, ACT, Australia beBeeDataEngineer Full time $95,000 - $155,000

    A key opportunity exists to contribute to a pivotal role in data engineering for an esteemed organization. Australian Citizens residing in Australia only need apply.Essential criteria must be met, and candidates are required to address these points explicitly in their response or application.Key Duties and Responsibilities1. The selected candidate will be...

  • Senior Data Engineering Lead

    2 weeks ago

    Canberra, ACT, Australia beBeeData Full time $149,739 - $173,174

    About the JobThe successful candidate will lead a team to manage NAPLAN data workflows, including collection, analysis, quality assurance and reporting.They will be responsible for developing project plans, managing risks and delivering projects on time to achieve organisational objectives.The role requires high level experience implementing and overseeing...

  • Lead Data Specialist

    1 week ago

    Canberra, ACT, Australia beBeeDataEngineer Full time $149,739 - $173,174

    Job OverviewA lead data engineer is needed to manage NAPLAN data workflows. This involves collecting data, analyzing it, and ensuring quality before reporting the results to various stakeholders.About the RoleThe lead data engineer will develop project plans, risk management strategies, and ensure delivery of NAPLAN projects in line with project...

  • IT Infrastructure Engineer

    2 weeks ago

    Canberra, ACT, Australia CDC Data Centres Full time

    About us:Established in 2007, CDC Data Centres is Australia's leading operator of sovereign, secure, world-class data centre facilities.Headquartered in Canberra with multiple facilities in the ACT, NSW and New Zealand, CDC's diversified operations provide secure data centre configurations that support co-location, containerised and hyperscale compute...

  • It Infrastructure Engineer

    1 week ago

    Canberra, ACT, Australia Cdc Data Centres Full time

    About us:Established in 2007, CDC Data Centres is Australia's leading operator of sovereign, secure, world-class data centre facilities.Headquartered in Canberra with multiple facilities in the ACT, NSW and New Zealand, CDC's diversified operations provide secure data centre configurations that support co-location, containerised and hyperscale compute...

  • Data Engineer

    3 weeks ago

    Canberra, ACT, Australia Compas Full time

    Compas Canberra, Australian Capital Territory, AustraliaCompas Canberra, Australian Capital Territory, Australia1 day ago Be among the first 25 applicantsGet AI-powered advice on this job and more exclusive features.Direct message the job poster from CompasData EngineerAre you an experienced Data Engineer with strong technical documentation and stakeholder...