Lead Data Engineer

2 weeks ago

Sydney, New South Wales, Australia ClearCompany Full time

Lead Data Engineer

Sydney, Australia

About the Role:

At ClearCompany, you will lead regulatory initiatives and drive innovation in data engineering, utilizing AI and automation to enhance platform efficiency for a leading financial services institution.

Key Responsibilities:

  • Lead the development and transformation of the regulatory reporting platform.
  • Collaborate across business and technical teams to deliver complex data solutions.
  • Implement and optimize data integration pipelines and data warehouse solutions.
  • Oversee the design and architecture of large-scale data solutions using AWS services.
  • Drive platform efficiency through automation and AI enhancements.
  • Ensure data governance and compliance with internal and external regulations.
  • Provide leadership and mentorship to data engineers, fostering a culture of innovation and excellence.

Required Skills and Experience:

  • Proficient in Spark, Hive, Presto, and big data storage formats (Parquet, ORC, Avro).
  • Experience with data pipeline orchestration tools such as Airflow.
  • Hands-on experience with data integration tools (Talend, DataStage, or Informatica).
  • Strong AWS experience, including EC2, S3, RDS, and EMR.
  • Expertise in data modeling techniques and data architecture best practices.
  • Knowledge of data lineage, technical metadata, data quality, and reconciliation processes.
  • Advanced skills in Spark SQL and Python for data automation.
  • Ability to tune and optimize data queries for performance.
  • Strong communication and negotiation skills.
  • Proven experience in leading cross-platform data solutions and complex projects.
  • Ability to mentor and lead technical teams.

  • Data Platform Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Rich Data Co Full time

    About RDCRich Data Co (RDC) delivers the future of credit today. We believe credit should be accessible, fair, inclusive, and sustainable. We are passionate about AI and developing new techniques that leverage traditional and non-traditional data to make accurate decisions in a clear and explainable way. Global leading financial institutions use our AI...

  • Data & ML Engineering Manager

    2 weeks ago

    Sydney, New South Wales, Australia Data & Ai Talent Australia Full time

    4 days ago Be among the first 25 applicantsWe've been exclusively engaged by a trailblazing Australian tech company that's rapidly scaling across the APAC region. Backed by strong funding and a proven product-market fit, they're doubling down on data, machine learning, and AI to supercharge their next phase of growth—and they're looking for the right...

  • Data & ML Engineering Manager

    2 weeks ago

    Sydney, New South Wales, Australia Data & Ai Talent Australia Full time

    4 days ago Be among the first 25 applicantsWe've been exclusively engaged by a trailblazing Australian tech company that's rapidly scaling across the APAC region. Backed by strong funding and a proven product-market fit, they're doubling down on data, machine learning, and AI to supercharge their next phase of growth—and they're looking for the right...

  • Data Engineering Lead

    2 weeks ago

    Sydney, New South Wales, Australia Talent Insights Group Full time

    Senior Data Consultant at Talent InsightsLocation: Sydney (Hybrid Work)Position Type: Full-Time, PermanentAn exciting opportunity has arisen for a hands-on, proactive Data Engineering Lead to join a leading Insurance company based in Sydney.In this role, you will be responsible for leading a team of 2-4 data engineers, working as part of a global team on...

  • Data Engineering Lead

    2 weeks ago

    Sydney, New South Wales, Australia Correlate Resources Full time

    Our client is a market leading business with a high profile and reputable brand within the Australian market. With a well-established analytics division focused on optimising both customer and operational initiatives they boast an energising, supportive and collaborative culture, and are looking for great people who want to be a part of their journey. With...

  • Lead Data Engineer

    2 weeks ago

    Sydney, New South Wales, Australia beBee Careers Full time

    **Lead Data Engineer:**Cloud Data Solutions ArchitectWe are seeking a skilled Data Engineer to lead our Data & Analytics Platforms strategy. As a senior manager, you will be responsible for setting and executing our ANZ Data & Analytics Platform strategy, delivering cloud-native data solutions, and leading digital transformation efforts.The ideal candidate...

  • Lead Data Engineer

    5 days ago

    Sydney, New South Wales, Australia tekFinder Full time

    Direct message the job poster from tekFinderWe work with innovative technology companies in Australia, helping them build great software engineering teams.Lead Data Engineer - ETL, Spark SQL, AWS - up to $215k package + bonusWe are recruiting a Lead Data Engineer to spearhead the transformation of a regulatory platform.The role is within the regulated...

  • Data Engineering Lead

    1 week ago

    Sydney, New South Wales, Australia Firesoft People Full time

    Job DescriptionWe are seeking a highly skilled Data Engineering Lead to join our team at Firesoft People.This is a key role in driving the success of our Data & Analytics Practice. The successful candidate will be responsible for leading platform engineering teams across multiple locations and driving collaborative cross-functional initiatives from strategy...

  • Lead Data Engineer

    2 weeks ago

    Sydney, New South Wales, Australia Pyramid Global Technologies Full time

    Lead Data Engineer Role at Pyramid Global TechnologiesWe are seeking a Lead Data Engineer to join our team in Sydney, requiring 10+ years of experience in Software Engineering, Cloud Computing, Databricks, and Big Data.Main Responsibilities:Data Engineering and Data Warehousing: Design, implement, and maintain scalable data architectures using cloud-based...

  • Lead Data Engineer

    3 weeks ago

    Sydney, New South Wales, Australia ClearCompany Full time

    Lead Data EngineerSydney, Australia$230,000 (including Super) + BonusAbout the Role:Join a high-performing Data Team at a leading financial services institution to lead regulatory initiatives and drive innovation in data engineering, utilizing AI and automation to enhance platform efficiency.Key Responsibilities:Lead the development and transformation of the...