Data Engineering Leader

5 days ago

Melbourne, Victoria, Australia beBeeEngineering Full time $104,000 - $130,000
Job Summary

We are seeking a Data Engineering Leader to join our mission-driven team. As a key member, you will design and architect scalable data pipelines, ensuring reliable and efficient data processing from multiple sources.

Your responsibilities will include leading the development of secure, cost-effective data infrastructure using AWS services and Terraform, implementing robust monitoring, alerting, and logging systems, and enhancing observability to minimize downtime and optimize performance.

You will also develop automated data governance strategies, including data lineage tracking, access control, masking, and encryption for GDPR compliance, and implement data governance processes for regulatory standards.

In addition, you will drive improvements in engineering best practices, including code quality, system design, documentation, and operational efficiency, and identify opportunities for innovative solutions to enhance data capabilities, solving performance bottlenecks and improving data reliability.

Key Responsibilities:

  • Design, architect, and optimize scalable ETL pipelines using AWS Glue, orchestrating complex workflows with Airflow for efficient data processing from multiple sources.
  • Lead development of secure, scalable, and cost-efficient data infrastructure using AWS services and Terraform, ensuring best practices in automation.
  • Collaborate with stakeholders to define strategic data initiatives, including architecture, real-time processing, and analytics enablement.
  • Enhance observability with robust monitoring, alerting, and logging systems, minimizing downtime and optimizing performance.
  • Develop automated data governance strategies, including data lineage tracking, access control, masking, and encryption for GDPR compliance.
  • Implement data governance processes for compliance with data lineage, classification, access control, GDPR, and other regulatory standards.
  • Architect and optimize data pipelines for Change Data Capture, reconciliation workflows, and reliability enhancements.
  • Drive improvements in engineering best practices, including code quality, system design, documentation, and operational efficiency.
  • Identify opportunities for innovative solutions to enhance data capabilities, solving performance bottlenecks, and improving data reliability.
Benefits
  • Unique exposure to large datasets handling 500 million records daily and 10 MB real-time input.
  • Champion engineering excellence, influencing global impact across software products.
  • Work with top talent using a vast AWS cloud native technology stack.
  • Access to 9,000 courses in our learning platform.
  • EAP access for family.
  • Lucrative annual bonuses.
  • Paid volunteer day.
  • Daily breakfast and snacks.
  • On-site massage Wednesdays.
  • In-house baristas providing coffee needs.
  • Weekly team lunches and happy hour Fridays.
  • Bustling office with up to 2 days work from home per week option.
  • Fun office environment.

We value everyone's unique contributions, driving our success. To incorporate diverse perspectives and experience, we never discriminate based on race, religion, national origin, gender or expression, sexual orientation, age, or marital status. We provide an equal workplace encouraging great participation and merit-based success.

  • Data Engineer

    1 week ago

    Melbourne, Victoria, Australia Tech & Data People Full time $104,000 - $130,878 per year

    Are you the kind of engineer who loves solving tricky data problems, automating the boring stuff, and making things run faster and smarter? This role puts you right in the middle of a cloud-first data environment where you'll have the freedom to design, build and optimise pipelines, bring systems together, and use DevOps to keep everything running...

  • Data Engineering Leader

    5 days ago

    Melbourne, Victoria, Australia beBeeCloud Full time $120,000 - $150,000

    Job OpportunityPartner with an organization to drive innovation in data and technology, accelerating business growth.As a key data engineering leader, you'll design, develop, and scale a cutting-edge cloud data platform on AWS. You will be hands-on in building data pipelines and infrastructure while providing expert guidance and mentoring.Key...

  • Data Engineering Leader

    6 days ago

    Melbourne, Victoria, Australia beBeeLeader Full time $150,000 - $180,000

    About the role:We're seeking a seasoned Data Engineering leader to join our team. This is an exciting opportunity to drive business growth and customer satisfaction through cutting-edge data technologies.The successful candidate will be responsible for designing, implementing, and maintaining highly available, scalable, and secure data pipelines and...

  • Data Engineering Leader

    2 weeks ago

    Melbourne, Victoria, Australia Bebeedata Full time

    Elevate our data strategy and engineering capabilities across teams.About the RoleWe are seeking a Principal Data Engineer to define and lead our data strategy, including data warehousing, ETL/ELT processes, and data platform architecture.You will partner with the security team to ensure data architecture and governance meet compliance, regulatory, and...

  • Data Engineering Leader

    1 week ago

    Melbourne, Victoria, Australia beBeeDataEngineering Full time $170,000 - $210,000

    Job Title: Data Engineering LeaderJob Description:You will be part of a high-performing team on a mission to build a cutting-edge data platform that powers everything we do.This role offers the opportunity to work on complex, large-scale data engineering projects and collaborate with stakeholders to define strategic data initiatives. You will lead the...

  • Data Engineering Leader

    7 days ago

    Melbourne, Victoria, Australia beBeeDataEngineering Full time $120,000 - $180,000

    Data Engineering LeaderWe are seeking an experienced professional to lead the development of our data engineering solutions. This is a unique opportunity to contribute to the growth and success of our organisation.The successful candidate will be responsible for designing, implementing, and maintaining scalable data pipelines leveraging automated testing,...

  • Data Platform Engineer Leader

    1 week ago

    Melbourne, Victoria, Australia beBeeEngineering Full time $200,000 - $250,000

    Senior Engineering Leader">We are seeking a driven and experienced Senior Engineering Leader to join our team.The ideal candidate will have a strong background in data engineering and be passionate about driving innovation through AI-powered engineering.You will lead a team of platform engineers and be responsible for developing scalable and sustainable...

  • Data Engineer Leader

    1 week ago

    Melbourne, Victoria, Australia Bebeedata Full time

    Senior Data Engineering PositionWe are seeking a highly skilled data engineer to design and implement robust, scalable, and secure data platforms. The successful candidate will have extensive experience in architecting and leading the development of cutting-edge data solutions.This is an excellent opportunity for a senior data engineer to leverage their...

  • Principal Data Engineer

    6 days ago

    Melbourne, Victoria, Australia Tech & Data People Full time $250,000 per year

    Principal Data Engineer – Melbourne – Hybrid - $250,000 - bleeding-edge Cloud & Data tech - enable next-gen analytics & AI Tech and Data People are working with a leading organisation that is driving large-scale digital and data transformation. We're looking for a Principal Data Engineer to design and deliver modern, secure, and scalable data...

  • Data Engineer Leader

    1 week ago

    Melbourne, Victoria, Australia beBeeData Full time $180,000 - $240,000

    Senior Data Engineering PositionWe are seeking a highly skilled data engineer to design and implement robust, scalable, and secure data platforms. The successful candidate will have extensive experience in architecting and leading the development of cutting-edge data solutions.This is an excellent opportunity for a senior data engineer to leverage their...