Strategic Data Engineering Lead

2 weeks ago

Perth, Western Australia beBeeDataEngineering Full time $160,000 - $190,000
Lead Data Engineer Position Overview

Our company is seeking a highly skilled Lead Data Engineer to join our team and play a key role in designing, building, and optimising data pipeline solutions on Databricks and related cloud platforms. The successful candidate will work closely with data scientists, analysts, and engineers to ensure our data infrastructure supports advanced analytics and business insights across industries.

This position requires a unique blend of technical expertise, leadership skills, and collaboration abilities. If you are passionate about leveraging big data technologies to drive business growth, we encourage you to apply for this exciting opportunity.

Key Responsibilities:
  • Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver effective pipeline solutions.
  • Contribute to Data Architecture and Solution Design, helping to build proofs of concepts.
  • Design, develop, and maintain robust ETL/ELT pipelines on Databricks, using AWS, Azure, and GCP tools and services to ingest, process, and transform large datasets.
  • Implement data validation, cleansing, and governance procedures to guarantee data quality, integrity, and security, including enforcing data standards and proactively addressing data quality issues.
  • Continuously improve the scalability, efficiency, and cost-effectiveness of data pipelines and identify opportunities to enhance performance, reliability, and cost-efficiency across data systems.
  • Monitor data pipeline performance and troubleshoot issues to ensure high data availability and consistency; leverage observability tools and best practices to maintain reliable pipelines.
  • Develop streaming or event-driven data processes for real-time analytics, leveraging frameworks like Apache Kafka and Spark Structured Streaming.
Required Skills and Qualifications:
  • Proficiency in Python for data engineering (including PySpark and libraries like pandas/Polars) and SQL for data querying and transformation.
  • Solid understanding of data warehousing concepts and dimensional data modeling (e.g. star schema, Kimball methodology).
  • Hands-on experience with relational database systems and SQL (e.g. SQL Server, PostgreSQL) and familiarity with NoSQL databases (e.g. MongoDB, Cassandra).
  • Strong experience designing and implementing ETL/ELT processes and integrating data from multiple sources.
  • Proven experience working with multiple cloud data platforms such as AWS, Azure, and GCP.
  • Expertise in Databricks and the Spark ecosystem for large-scale data processing.
  • Familiarity with data pipeline orchestration and automation tools and with CI/CD pipelines for deploying data workflows.
What We Offer:
  • A competitive salary package, share plan, company bonuses, value-based recognition awards, referral bonus.
  • Career coaching, global career opportunities, non-linear career paths, internal development programs for management and technical leadership.
  • Complex projects, rotations, internal tech communities, training, certifications, coaching, online learning platform subscriptions.
  • Flexible work arrangements and employee assistance program.
  • Global wellbeing program, access to wellbeing apps.

  • Data Engineering Lead

    2 weeks ago

    Perth, Western Australia beBeeEngineering Full time $120,000 - $150,000

    About the RoleWe are seeking an experienced and motivated Data Engineer to lead our data capability forward. With a strong track record of driving innovative solutions, you will play a key role in shaping our data strategy.Key Responsibilities:Design and implement robust, high-performing data solutions that meet business needs.Manage and enhance...

  • Data Engineering Lead for Sustainable Mining

    1 week ago

    Perth, Western Australia beBeeMineclosure Full time $120,000 - $180,000

    Drive Innovation in Mine Closure and RehabilitationThis is a unique opportunity to spearhead the design, delivery, and scaling of next-generation data engineering and architecture solutions that power mine closure and rehabilitation outcomes.Key ResponsibilitiesYou will take ownership of large-scale data projects, shaping strategy and architecture, managing...

  • Lead Data Engineer

    1 week ago

    Perth, Western Australia Endava Full time

    Company DescriptionTechnology is our how. And people are our why. For over two decades, we have been harnessing technology to drive meaningful change.By combining world-class engineering, industry expertise and a people-centric mindset, we consult and partner with leading brands from various industries to create dynamic platforms and intelligent digital...

  • Lead Data Engineer

    2 weeks ago

    Perth, Western Australia Endava Full time

    Overview Endava Perth, Western Australia, Australia is seeking a Lead Data Engineer specialising in Databricks to design, build, and optimise data pipeline solutions on Databricks and related cloud platforms. You will work closely with data scientists, analysts, and engineers to ensure our data infrastructure supports advanced analytics and business...

  • Lead Data Engineer

    2 weeks ago

    Perth, Western Australia Endava Full time

    OverviewEndava Perth, Western Australia, Australia is seeking a Lead Data Engineer specialising in Databricks to design, build, and optimise data pipeline solutions on Databricks and related cloud platforms. You will work closely with data scientists, analysts, and engineers to ensure our data infrastructure supports advanced analytics and business insights...

  • Lead Electrical Engineer

    3 weeks ago

    Perth, Western Australia BEC Engineering Full time

    Our CompanyBEC Engineering is a well-established and respected Electrical, Power and Control System Engineering firm based in West Perth.With a strong pipeline of work, we are looking for aLead Electrical Engineerwith 10+ years experience to join our team.About the RoleReporting to the Electrical Engineering Manager, this role forms part of the core...

  • Data Engineer

    7 days ago

    Perth, Western Australia Deloitte Full time

    Salary packaging – to suit your personal and financial circumstances Flexible work arrangements – work in a way that suits you best Rewards platform – your hard work won't go unnoticed at Deloitte Open to candidates across Australia About the Role: Are you curious about the world of people data?Do you love to work with data which is dynamic and always...

  • Senior Data Engineer

    2 weeks ago

    Perth, Western Australia beBeeData Full time $200,000 - $250,000

    Job Title: Data Architecture Strategist">Job Description:">We are seeking an experienced data professional to lead the design and implementation of our data architecture strategy.">This is a senior-level position that requires a strong technical background and excellent problem-solving skills. You will work closely with cross-functional teams to drive the...

  • Strategic Data Governance Lead

    2 weeks ago

    Perth, Western Australia beBeeDataGovernance Full time $122,672 - $130,758

    Job TitleWe are seeking an experienced professional to lead the development and implementation of data governance strategies across our organisation. This is a critical role that will play a key part in shaping our data management practices and ensuring they are aligned with our corporate goals.The successful candidate will have significant experience in...

  • Principal Data Centre Technical Lead

    2 weeks ago

    Perth, Western Australia Bebeeelectricalengineer Full time

    Our organization seeks an exceptional electrical engineer to lead complex projects and drive business growth.The OpportunityThis role offers the chance to make a significant impact on our presence in the data centre space while working in a business that values innovation, inclusion, and purpose-driven work.Lead high-profile data center projects, managing...