Data Pipeline Specialist

1 week ago

Sydney, New South Wales, Australia beBeeDevops Full time $80,000 - $130,000
Job Overview

We are seeking a skilled DevOps consultant to join our team on a long-term contract in Sydney. The ideal candidate will have unrestricted work rights in Australia.

Key Responsibilities:

  • Design and develop scalable data pipelines using Azure Databricks, PySpark, and SQL.
  • Implement structured streaming, Delta Lake, and Unity Catalog.
  • Optimize Spark jobs and manage cluster performance.
  • Experience in production support environment.
  • Expertise with Azure Databricks solutions.
  • Build and maintain CI/CD pipelines using Azure DevOps, GitHub, Jenkins, and Terraform.
  • Automate deployment of notebooks, clusters, secrets, and data workflows.
  • Manage version control, branching strategies, and code reviews.
  • BODS ETL skills are an advantage as the scope includes supporting BODS ETL framework-based Oracle database reporting systems.
  • Collaborate with cross-functional teams for requirement gathering and solution design.
  • Document technical specifications, data dictionaries, and architecture blueprints.

Desirable skills: MLflow, Power BI, Databricks Dashboards, Delta Live Tables, Agile, Jira, Test Management Tools.

Note: This job requires full unrestricted work rights in Australia.

  • Data Pipeline Specialist

    2 weeks ago

    Sydney, New South Wales, Australia beBeeDataPipeline Full time $200,000 - $250,000

    Key Role: Data Pipeline SpecialistWe are seeking a skilled data pipeline specialist to join our team.Design, optimize, and scale data pipelines using software engineering best practices (testing, version control, automation).Structure dbt projects and layers (staging, intermediate, marts) to ensure clarity, reusability, and business alignment.Embed...

  • Senior Data Pipeline Specialist

    1 week ago

    Sydney, New South Wales, Australia beBeeData Full time $120,000 - $180,000

    Lead Data EngineerThis is a 6-month project in Sydney with the possibility of extension.About the Role:The ideal candidate will have extensive experience in ADF and SSIS development, GIT, and JSON. They will be responsible for developing data pipelines, performing code reviews, issue resolution, and mentoring junior developers.Key Responsibilities:Develop...

  • Data Pipeline Specialist

    2 weeks ago

    Sydney, New South Wales, Australia beBeeAutomation Full time $140,000 - $170,000

    Systems ArchitectWe are a global financial trading firm with a drive to deeply understand the world around us and make optimal business decisions every day.Your role involves performing support, deployment, and automation tasks related to data pipelines throughout the Asia Pacific region as part of our trading technology team.This role is integral to our...

  • Data Pipeline Architect

    2 weeks ago

    Sydney, New South Wales, Australia Bebeeintegration Full time

    Job Title: Data Integration SpecialistWe are on a mission to be a force for good, through our people, products and purpose.We seek individuals who are passionate, able to work at pace, comfortable with ambiguity, and committed to making a positive difference.Key Responsibilities:Design, build and optimize pipelines using AWS Glue, dbt, Airflow or similar...

  • Data Pipeline Specialist

    2 weeks ago

    Sydney, New South Wales, Australia beBeeDataEngineer Full time $130,500 - $164,600

    Job SummaryWe are seeking an experienced Data Engineer to develop and maintain robust data pipelines and infrastructure for analytics, machine learning, and reporting platforms.The ideal candidate will have a strong proficiency in cloud data warehouses, proficient SQL and Python coding skills, and experience developing Airflow or other open-source data...

  • Data Pipelines Specialist

    1 week ago

    Sydney, New South Wales, Australia beBeeDataEngineer Full time $136,000 - $164,000

    About our ideal candidateWe are seeking a skilled professional to join our team in Sydney, Australia. This individual will be responsible for developing core data pipelines and tools that turn real-world, multi-sensor data into actionable intelligence — powering systems that operate in the world's most complex and high-stakes environments.This is a unique...

  • Senior Data Pipeline Specialist

    2 weeks ago

    Sydney, New South Wales, Australia beBeeData Full time $120,000 - $175,000

    Are you a seasoned data professional looking for your next challenge? We have an exciting opportunity for a Data Engineer to join our team in a large-scale program. As a key member of our technical team, you will be responsible for designing, building, and maintaining data pipelines in an Azure environment.This role requires strong expertise in Databricks...

  • Senior Data Pipeline Specialist

    1 week ago

    Sydney, New South Wales, Australia beBeeDataEngineer Full time $150,000 - $180,000

    Data EngineerAbout the RoleWe're seeking a skilled data engineer to manage our data pipelines and create integrations with various platforms, vendors, and third-party data providers. This is a hands-on role where you'll understand business needs to create efficient data flows within technical constraints.The ideal candidate will have experience in designing,...

  • Expert Data Pipeline Architect

    1 week ago

    Sydney, New South Wales, Australia beBeeDataEngineer Full time $120,000 - $160,000

    As a highly skilled Data Engineer, you will play a critical role in designing, building, and maintaining scalable data pipelines using Databricks, PySpark, and Spark SQL.The ideal candidate will have strong expertise in Azure Synapse, with experience in developing data integration and transformation solutions on spark and sql pools.We are seeking a talented...

  • Azure Cloud Engineer

    2 weeks ago

    Sydney, New South Wales, Australia beBeeDevOps Full time $180,000 - $250,000

    DevOps Engineer RoleWe are seeking a highly skilled DevOps Specialist to join our team. The successful candidate will have expertise in designing and developing scalable data pipelines using Azure Databricks, PySpark, and SQL.Main ResponsibilitiesData Pipeline Development: Implement structured streaming, Delta Lake, and Unity Catalog solutions.Optimization...