Principal Data Pipeline Specialist

4 days ago

Sydney, New South Wales, Australia beBeeDataEngineer Full time $80,000 - $120,000
Job Overview

We're seeking a talented Data Engineer to support the development and maintenance of modern data pipelines. In this role, you'll gain hands-on experience with tools like DBT, Airflow, Snowflake, and Databricks while working alongside experienced engineers to design data models, improve data quality, and document solutions.

  • Data Engineering Support: Assist in the development, maintenance, and monitoring of data pipelines using modern data stack tools under senior guidance.
  • Data Analysis & Solution Design: Contribute to data analysis and solution prototyping activities as part of a team; work with other engineers in designing new data models and enhancements, documenting learning and approaches.
  • Technical Skills Development: Gain hands-on experience using data pipeline and ETL tools (DBT, Airflow, Python) through active participation in team projects; learn and support processes related to cloud data warehouse deployments (development, testing, production); work with Snowflake and Databricks environments, assisting with Unity Catalog, Delta Live Tables & Spark under supervision.
  • Collaboration & Communication: Collaborate closely with data engineers, analysts, and business stakeholders to gather requirements and clarify technical specifications; communicate technical challenges or questions, escalating appropriately for senior support.
  • Documentation: Maintain clear and concise documentation for data engineering tasks, including updated records of pipelines, workflows, and technical solutions; assist in preparing user and process documentation as part of project deliverables.
  • Data Governance & Quality: Support the implementation of data governance policies by monitoring data quality and flagging issues; run routine data validation checks and report anomalies to the senior engineering team.
  • Continuous Learning: Stay updated on industry trends and best practices in data engineering and cloud technologies; actively seek mentoring and learning opportunities within the team and through provided resources.
  • Performance & Optimisation: Observe and learn from seniors on optimising data pipelines and improving process performance; assist in routine performance monitoring tasks and suggest incremental improvements as knowledge grows.
Required Qualifications
  • A bachelor's degree in computer science, data science, IT, or a related field.
  • Basic SQL skills and an understanding of relational databases.
  • Familiarity with Python and an interest in data engineering tools such as DBT or Airflow.
  • Exposure to cloud data platforms, ideally Snowflake or Databricks.
  • Understanding of data modelling concepts and data governance principles.
  • Strong problem-solving skills and attention to detail.
Benefits
  • A supportive and collaborative work environment.
  • Flexible working arrangements designed to promote work-life balance.
  • Comprehensive wellbeing programs offering support for medical, mental health, and financial wellbeing.
  • Generous parental leave policies.
  • Additional loyalty leave after three years of service.

  • Principal Data Pipeline Specialist

    3 days ago

    Sydney, New South Wales, Australia beBeeData Full time $123,444 - $144,368

    Data Engineering RoleASIC is seeking a skilled Data Engineer to join our team. This critical role will involve designing and maintaining robust data ingestion pipelines, managing secure and efficient data transfers into ASIC's systems, and ensuring timely and accurate delivery of data to support core business functions.

  • Expert Data Pipeline Specialist

    3 days ago

    Sydney, New South Wales, Australia beBeeData Full time $100,000 - $150,000

    Job Summary:We are seeking an experienced Data Pipeline Specialist to play a vital role in ensuring the quality and integrity of our data pipelines.The ideal candidate will have 5+ years of expertise in SQL, Control-M, Unix, and data warehousing.Responsibilities Include:Design and ExecutionDesign, develop, and execute test plans and test cases for ETL...

  • Data Pipeline Specialist

    7 days ago

    Sydney, New South Wales, Australia beBeeDataEngineer Full time

    About the RoleDrive's Data Operations team is seeking an ETL Engineer to help us achieve our vision of shaping the future of automotive in Australia. As a key member of this team, you will be responsible for designing, implementing, and maintaining efficient data pipelines that drive business value.The ideal candidate will have 5+ years of experience with...

  • Data Pipeline Specialist

    2 days ago

    Sydney, New South Wales, Australia beBeeDevopser Full time $150,000 - $170,000

    Job DescriptionAs a DevOps Engineer, you will play a vital role in our trading technology team. You will be responsible for performing daily monitoring and market pre-open system checks utilizing automation, developing new tools and utilities to help build data pipelines and studies, and becoming a subject matter expert in our in-house systems and data...

  • Data Pipeline Specialist

    9 hours ago

    Sydney, New South Wales, Australia beBeeDataEngineer Full time $120,000 - $270,000

    Job Title: Data Engineer AssociateWe are looking for a highly skilled and motivated Data Engineer Associate to join our team. As a key member of our data engineering group, you will play a crucial role in designing, developing, and maintaining our modern data pipelines. You will be working closely with experienced engineers to develop solutions that improve...

  • Data Pipeline Specialist

    2 days ago

    Sydney, New South Wales, Australia beBeeTechnology Full time $150,000 - $170,000

    Job Description:Susquehanna is a global privately held financial trading firm with a unique approach to making business decisions every day.Their trading technology team performs support, deployment and automation tasks related to the management of data pipelines throughout the Asia Pacific region.Daily monitoring and market pre-open system checks are...

  • Data Pipeline Specialist

    2 days ago

    Sydney, New South Wales, Australia beBeeData Full time $150,000 - $200,000

    Technical Support EngineerSig-Susquehanna is a global financial trading firm that utilizes collective wisdom and relentless problem-solving to drive business decisions.Your Role:As part of our trading technology team, you will be responsible for performing support, deployment, and automation tasks related to data pipelines in the Asia Pacific region. You...

  • Data Pipeline Specialist

    1 day ago

    Sydney, New South Wales, Australia beBeeDataEngineer Full time $100,000 - $140,000

    Data Engineering RoleWe are seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and maintaining robust data pipelines to support data ingestion, transformation, and integration from various sources.The ideal candidate will have experience with Kafka, strong knowledge of AWS...

  • Data Pipeline Specialist

    4 days ago

    Sydney, New South Wales, Australia beBeeData Full time $100,000 - $150,000

    Job OpportunityWe are seeking a highly skilled and motivated Devops Engineer to join our trading technology team. This role offers a unique opportunity to work on cutting-edge data pipelines and systems, collaborating closely with developers, traders, and quant researchers.Key Responsibilities:Perform daily monitoring and market pre-open system checks...

  • Principal Cloud

    4 weeks ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time

    Principal Cloud (Azure) Services Sales SpecialistJoin to apply for the Principal Cloud (Azure) Services Sales Specialist role at NTT DATA, Inc.Principal Cloud (Azure) Services Sales Specialist4 days ago Be among the first 25 applicantsJoin to apply for the Principal Cloud (Azure) Services Sales Specialist role at NTT DATA, Inc.Get AI-powered advice on this...