Lead Data Engineer

4 days ago


Sydney, New South Wales, Australia ClearCompany Full time

Lead Data Engineer

Sydney, Australia

$230,000 (including Super) + Bonus

About the Role:
Join a high-performing Data Team at a leading financial services institution to lead regulatory initiatives and drive innovation in data engineering, utilizing AI and automation to enhance platform efficiency.

Key Responsibilities:

  1. Lead the development and transformation of the regulatory reporting platform.
  2. Collaborate across business and technical teams to deliver complex data solutions.
  3. Implement and optimize data integration pipelines and data warehouse solutions.
  4. Oversee the design and architecture of large-scale data solutions using AWS services.
  5. Drive platform efficiency through automation and AI enhancements.
  6. Ensure data governance and compliance with internal and external regulations.
  7. Provide leadership and mentorship to data engineers, fostering a culture of innovation and excellence.
Required Skills and Experience:
  1. Proficient in Spark, Hive, Presto, and big data storage formats (Parquet, ORC, Avro).
  2. Experience with data pipeline orchestration tools such as Airflow.
  3. Hands-on experience with data integration tools (Talend, DataStage, or Informatica).
  4. Strong AWS experience, including EC2, S3, RDS, and EMR.
  5. Expertise in data modeling techniques and data architecture best practices.
  6. Knowledge of data lineage, technical metadata, data quality, and reconciliation processes.
  7. Advanced skills in Spark SQL and Python for data automation.
  8. Ability to tune and optimize data queries for performance.
  9. Strong communication and negotiation skills.
  10. Proven experience in leading cross-platform data solutions and complex projects.
  11. Ability to mentor and lead technical teams.
If you feel you possess the relevant skills to the above, please respond with most updated CV
#J-18808-Ljbffr

  • Sydney, New South Wales, Australia Talent Insights Group Full time

    Senior Data Consultant at Talent InsightsLocation: Sydney (Hybrid Work)Position Type: Full-Time, PermanentAn exciting opportunity has arisen for a hands-on, proactive Data Engineering Lead to join a leading Insurance company based in Sydney.In this role, you will be responsible for leading a team of 2-4 data engineers, working as part of a global team on...


  • Sydney, New South Wales, Australia HCLTech - Australia and New Zealand Full time

    As a highly skilled professional, you will lead the Data Engineering team in HCLTech Australia and New Zealand, driving strategic initiatives that promote excellence and technical development. Your role will ensure seamless end-to-end delivery of complex data engineering projects and programmes for clients and internal products.The ideal candidate will have...

  • Data Engineer Lead

    5 days ago


    Sydney, New South Wales, Australia SiteMinder Full time

    SiteMinder is at the forefront of hotel commerce, and we're looking for a skilled Data Engineer Lead to drive our data transformation program. As a key member of our engineering team, you'll shape our data engineering strategy to deliver scalable, real-time, and high-quality data pipelines.What You'll Do:Elevate our data infrastructure to support...


  • Sydney, New South Wales, Australia ResMed Inc Full time

    About ResMed IncAt ResMed, we are a global leader in connected devices and digital health. Our mission is to create innovative solutions that transform the lives of millions of people worldwide.The Analytics Engineering TeamWe are seeking an experienced Data Engineering Lead to join our Analytics Engineering team within Global Technology Solutions (GTS). As...

  • Lead Data Engineer

    7 days ago


    Sydney, New South Wales, Australia Pyramid Global Technologies Full time

    Job Description for Lead Data Engineer in Sydney.Experience: 10+ years in Software Engineering, Cloud Computing, Databricks, and Big Data.Key Responsibilities:Data EngineeringData WarehousingSolution DesignData AnalysisVisualizationData ModelingProgramming Languages: Python, Scala, SQLCloud Technologies: Azure and DatabricksBig Data Technologies: Cloudera...


  • Sydney, New South Wales, Australia Mantel Group Pty Ltd Full time

    About Mantel GroupWe are an Australian-owned technology consulting business with a strong presence in Cloud, Digital, Data, Delivery & Security.Our company values diversity and innovation, fostering a culture where talented individuals can grow and thrive.As a key player in the industry, we strive to deliver exceptional solutions that meet our clients'...


  • Sydney, New South Wales, Australia One51 | Data & Analytics Consultancy Full time

    We are seeking a highly skilled Azure Data Engineer and Data Scientist to join our team at One51 | Data & Analytics Consultancy. In this role, you will be responsible for designing and developing the data architecture, data models, and standards for various Data Integration & Data Warehouse projects in Azure.Your Key Responsibilities Will Include:Designing...


  • Sydney, New South Wales, Australia One51 | Data & Analytics Consultancy Full time

    We are looking for an experienced Cloud Data Engineer to join our team at One51 | Data & Analytics Consultancy. In this role, you will be responsible for designing and developing the data architecture, data models, and standards for various Data Integration & Data Warehouse projects in Azure.Key Responsibilities:Developing ETL pipelines in and out of a data...

  • Lead Data Engineer

    4 days ago


    Sydney, New South Wales, Australia Paxus - Technology + Digital Talent Full time

    Permanent Opportunity open to interstate candidatesGenerous remunerationFlexible Working ArrangementOur client is looking for a Lead Data Engineer to shape their data platform and strategy. You will be the leader of making technical decisions, ensuring operational excellence, best engineering practices, and strategic vision across multiple projects.Key...

  • Data Engineering Lead

    4 hours ago


    Sydney, New South Wales, Australia Macquarie Bank Limited Full time

    Join our Group Treasury Transformation Liquidity Technology team as a Lead Data Engineer. We carry an exciting program of work to deliver our next-generation Liquidity Platform. This is an excellent opportunity for someone with a keen interest in Finance (Group Treasury), Data, and Technology.At Macquarie, our advantage is bringing together diverse people...

  • Lead Data Engineer

    12 hours ago


    Sydney, New South Wales, Australia Macquarie Bank Limited Full time

    Join our Customer Intelligence Engine (CIE) Technology team and help us build the best technology business in financial services. This is a great opportunity to be in a leadership as well as hands-on role while working with a team of talented engineers in developing the next generation Customer Intelligence Engine.At Macquarie, our advantage is bringing...


  • Sydney, New South Wales, Australia Infosys Singapore & Australia Full time

    Job DescriptionWe are seeking an experienced Data Engineer to lead our team in creating innovative data products. As a key member of our organization, you will be responsible for defining business requirements and working closely with engineering teams to build and test our data products. Your expertise in AWS Data Factory, Data Lake, Synapse, and Azure SQL...


  • Sydney, New South Wales, Australia Lead Group Full time

    Overview:We are seeking a highly skilled Senior HVAC Engineer Lead to join our team at Lead Group. The successful candidate will be responsible for managing multiple and varied installation projects from tender stage to completion.

  • Data Engineer Lead

    4 days ago


    Sydney, New South Wales, Australia Endava Limited Full time

    Company OverviewWe are a global company that harnesses technology to drive meaningful change. Our team combines world-class engineering, industry expertise, and a people-centric mindset to create dynamic platforms and intelligent digital experiences.We consult and partner with leading brands from various industries to transform businesses and drive...


  • Sydney, New South Wales, Australia Tbwa ChiatDay Inc Full time

    We're seeking a highly skilled data professional to join our Technology team and drive our data engineering strategy and foster a culture of innovation and collaboration.The successful candidate will have a proven track record of leading and scaling data engineering teams, and will be able to shape and execute a data roadmap that balances innovation with...


  • Sydney, New South Wales, Australia Lion Inc Full time

    About LionAt Lion, we're shaping the future of data-driven decision making. Our team is passionate about innovation and growth.Job SummaryWe're seeking a Senior AWS Data Engineer to expand and optimise our Data & Analytics platform. As a key member of our team, you'll drive the ingestion, transformation, and provisioning of data, empowering our Insights...


  • Sydney, New South Wales, Australia Macquarie Bank Limited Full time

    About UsMackquarie is a global financial services group operating in 34 markets with 55 years of unbroken profitability. Our diverse team of talented engineers works together to develop the next generation Customer Intelligence Engine, a cutting-edge technology that processes large volumes of data from various sources.As a Lead Data Engineer at Macquarie,...

  • Data Engineer Lead

    4 days ago


    Sydney, New South Wales, Australia Tech Mahindra Full time

    Job SummaryWe are seeking an experienced Big Data Specialist to lead our data engineering efforts. The ideal candidate will have a strong background in designing and implementing scalable data pipelines and workflows on the Databricks platform.Key ResponsibilitiesDesign, build, and maintain complex data pipelines and workflows on the Databricks...


  • Sydney, New South Wales, Australia Linktree Full time

    We're looking for a highly skilled Senior Data Engineer Lead to join our Data Platform team at Linktree. In this role, you will play a key part in building the architecture and tools that power our data analytics and reporting functions.The successful candidate will have 2+ years of experience in analytics engineering, data engineering, or a related field....


  • Sydney, New South Wales, Australia Macquarie Bank Limited Full time

    Job DescriptionThe role of Senior Data Engineering Lead at Macquarie Bank Limited involves leading and mentoring a team of data engineers to architect, build, and maintain scalable and efficient data pipelines. This includes developing and implementing best practices for data organisation and storage, designing and implementing real-time data streaming...