Data Pipeline Specialist

24 hours ago

Sydney, New South Wales, Australia beBeeAutomation Full time $140,000 - $170,000
Systems Architect

We are a global financial trading firm with a drive to deeply understand the world around us and make optimal business decisions every day.

Your role involves performing support, deployment, and automation tasks related to data pipelines throughout the Asia Pacific region as part of our trading technology team.

This role is integral to our quant data development team, collaborating closely with developers, traders, and quant researchers.

The position offers daily operations, automation, and developing new tools to improve monitoring, management, and deployment of market data pipelines mainly on Linux operating systems.

Key Responsibilities

  • Daily monitoring and market pre-open system checks using automation.
  • Developing new tools and utilities to help build data pipelines and studies.
  • Becoming a subject matter expert in our in-house systems and data pipelines across market data and static/reference data.
  • Building and using automation tools/scripting to automate tasks ensuring system performance, reliability, and accuracy.
  • Providing a link between developers, traders, and in-house technology teams for smooth information flow regarding system issues/changes, market data, and connectivity.
  • Writing and maintaining technical and user documentation for systems and procedures.
  • Identifying opportunities to improve system stability, reliability, and the trading environment through technology enhancements.

Requirements

  • Degree in Computer Science, Mathematics, or related technical discipline.
  • Proficiency in Python.
  • Effective communication skills with technical and non-technical staff.
  • Ability to work under demanding deadlines with changing priorities in a trading environment.
  • Understanding of financial markets, instruments, and market data concepts, especially in equities and derivatives.

Desired Technical Experience

  • Application support, deployment, and monitoring experience.
  • Experience with data pipelines and processing.
  • Proficiency in Linux.
  • Experience with git, GitLab, CI/CD pipelines.
  • Experience with Jenkins, Airflow.

  • Data Pipeline Specialist

    7 days ago

    Sydney, New South Wales, Australia beBeePipeline Full time $140,000 - $180,000

    Job Opportunity: Data Pipeline SpecialistWe are seeking a highly skilled Data Pipeline Specialist to join our team. The successful candidate will be responsible for designing, implementing, and maintaining robust data pipelines that integrate data from diverse sources.The ideal candidate will have strong proficiency in SQL and Python for data manipulation,...

  • Data Pipeline Specialist

    3 days ago

    Sydney, New South Wales, Australia beBeeDataPipeline Full time $200,000 - $250,000

    Key Role: Data Pipeline SpecialistWe are seeking a skilled data pipeline specialist to join our team.Design, optimize, and scale data pipelines using software engineering best practices (testing, version control, automation).Structure dbt projects and layers (staging, intermediate, marts) to ensure clarity, reusability, and business alignment.Embed...

  • Expert Data Pipeline Specialist

    2 weeks ago

    Sydney, New South Wales, Australia beBeeData Full time $100,000 - $150,000

    Job Summary:We are seeking an experienced Data Pipeline Specialist to play a vital role in ensuring the quality and integrity of our data pipelines.The ideal candidate will have 5+ years of expertise in SQL, Control-M, Unix, and data warehousing.Responsibilities Include:Design and ExecutionDesign, develop, and execute test plans and test cases for ETL...

  • Senior Data Pipeline Specialist

    2 hours ago

    Sydney, New South Wales, Australia beBeeData Full time $120,000 - $180,000

    Lead Data EngineerThis is a 6-month project in Sydney with the possibility of extension.About the Role:The ideal candidate will have extensive experience in ADF and SSIS development, GIT, and JSON. They will be responsible for developing data pipelines, performing code reviews, issue resolution, and mentoring junior developers.Key Responsibilities:Develop...

  • Data Pipeline Architect

    3 days ago

    Sydney, New South Wales, Australia Bebeeintegration Full time

    Job Title: Data Integration SpecialistWe are on a mission to be a force for good, through our people, products and purpose.We seek individuals who are passionate, able to work at pace, comfortable with ambiguity, and committed to making a positive difference.Key Responsibilities:Design, build and optimize pipelines using AWS Glue, dbt, Airflow or similar...

  • Data Pipeline Specialist

    4 days ago

    Sydney, New South Wales, Australia beBeeDataEngineer Full time $130,500 - $164,600

    Job SummaryWe are seeking an experienced Data Engineer to develop and maintain robust data pipelines and infrastructure for analytics, machine learning, and reporting platforms.The ideal candidate will have a strong proficiency in cloud data warehouses, proficient SQL and Python coding skills, and experience developing Airflow or other open-source data...

  • Data Pipeline Specialist

    5 days ago

    Sydney, New South Wales, Australia beBeeETLEngineer Full time $120,000 - $140,000

    About this RoleWe are seeking an experienced ETL engineer to shape the future of automotive in Australia by providing authoritative reviews, detailed comparisons, and innovative commerce solutions.Your Key ResponsibilitiesDesign, implement, and continuously expand data pipelines by performing extraction, transformation, and loading activities.Work with...

  • Data Pipeline Specialist

    2 weeks ago

    Sydney, New South Wales, Australia beBeeDevopser Full time $150,000 - $170,000

    Job DescriptionAs a DevOps Engineer, you will play a vital role in our trading technology team. You will be responsible for performing daily monitoring and market pre-open system checks utilizing automation, developing new tools and utilities to help build data pipelines and studies, and becoming a subject matter expert in our in-house systems and data...

  • Data Pipeline Specialist

    1 week ago

    Sydney, New South Wales, Australia beBeeDataEngineer Full time $120,000 - $270,000

    Job Title: Data Engineer AssociateWe are looking for a highly skilled and motivated Data Engineer Associate to join our team. As a key member of our data engineering group, you will play a crucial role in designing, developing, and maintaining our modern data pipelines. You will be working closely with experienced engineers to develop solutions that improve...

  • Data Pipeline Architect

    5 days ago

    Sydney, New South Wales, Australia beBeeIntegration Full time $150,000 - $200,000

    Job Title: Data Integration SpecialistWe are on a mission to be a force for good, through our people, products and purpose.We seek individuals who are passionate, able to work at pace, comfortable with ambiguity, and committed to making a positive difference.Key Responsibilities:Design, build and optimize pipelines using AWS Glue, dbt, Airflow or similar...