Data Pipeline Specialist
2 weeks ago
This is a hands-on role with a focus on building, managing and optimising scalable data pipelines. You will play a key role in shaping our data platform and ensuring we meet the growing demands of our business.
Key Responsibilities:- Design and deploy data pipelines using Kafka to support business growth.
- Build and maintain producer/consumer libraries for seamless data integration.
- Monitor and improve Kafka performance, latency and reliability.
- Collaborate with cross-functional teams to integrate streaming pipelines into research workflows.
- Implement robust security measures for Kafka, including access control, authentication and encryption.
- Support disaster recovery planning and ensure production stability.
- Deliver high-quality data to cloud storage solutions like S3 and support lakehouse ingestion.
- Document systems and share best practices for efficient data management.
- Minimum 3 years of experience in data engineering.
- Hands-on expertise with Apache Kafka in production environments.
- Solid understanding of Kafka architecture and its components.
- Experience deploying, scaling and monitoring Kafka clusters.
- Proficiency in programming languages such as Python, Java, Scala or Bash.
- Comfortable working in Linux environments.
-
Data Pipeline Specialist
2 weeks ago
Sydney, New South Wales, Australia beBeeDataPipeline Full time $200,000 - $250,000Key Role: Data Pipeline SpecialistWe are seeking a skilled data pipeline specialist to join our team.Design, optimize, and scale data pipelines using software engineering best practices (testing, version control, automation).Structure dbt projects and layers (staging, intermediate, marts) to ensure clarity, reusability, and business alignment.Embed...
-
Senior Data Pipeline Specialist
1 week ago
Sydney, New South Wales, Australia beBeeData Full time $120,000 - $180,000Lead Data EngineerThis is a 6-month project in Sydney with the possibility of extension.About the Role:The ideal candidate will have extensive experience in ADF and SSIS development, GIT, and JSON. They will be responsible for developing data pipelines, performing code reviews, issue resolution, and mentoring junior developers.Key Responsibilities:Develop...
-
Data Pipeline Specialist
1 week ago
Sydney, New South Wales, Australia beBeeAutomation Full time $140,000 - $170,000Systems ArchitectWe are a global financial trading firm with a drive to deeply understand the world around us and make optimal business decisions every day.Your role involves performing support, deployment, and automation tasks related to data pipelines throughout the Asia Pacific region as part of our trading technology team.This role is integral to our...
-
Data Pipeline Architect
2 weeks ago
Sydney, New South Wales, Australia Bebeeintegration Full timeJob Title: Data Integration SpecialistWe are on a mission to be a force for good, through our people, products and purpose.We seek individuals who are passionate, able to work at pace, comfortable with ambiguity, and committed to making a positive difference.Key Responsibilities:Design, build and optimize pipelines using AWS Glue, dbt, Airflow or similar...
-
Data Pipeline Specialist
2 weeks ago
Sydney, New South Wales, Australia beBeeDataEngineer Full time $130,500 - $164,600Job SummaryWe are seeking an experienced Data Engineer to develop and maintain robust data pipelines and infrastructure for analytics, machine learning, and reporting platforms.The ideal candidate will have a strong proficiency in cloud data warehouses, proficient SQL and Python coding skills, and experience developing Airflow or other open-source data...
-
Data Pipeline Specialist
2 weeks ago
Sydney, New South Wales, Australia beBeeETLEngineer Full time $120,000 - $140,000About this RoleWe are seeking an experienced ETL engineer to shape the future of automotive in Australia by providing authoritative reviews, detailed comparisons, and innovative commerce solutions.Your Key ResponsibilitiesDesign, implement, and continuously expand data pipelines by performing extraction, transformation, and loading activities.Work with...
-
Data Pipeline Architect
2 weeks ago
Sydney, New South Wales, Australia beBeeIntegration Full time $150,000 - $200,000Job Title: Data Integration SpecialistWe are on a mission to be a force for good, through our people, products and purpose.We seek individuals who are passionate, able to work at pace, comfortable with ambiguity, and committed to making a positive difference.Key Responsibilities:Design, build and optimize pipelines using AWS Glue, dbt, Airflow or similar...
-
Data Pipeline Strategist
2 weeks ago
Sydney, New South Wales, Australia beBeeData Full time $90,000 - $120,000Job OverviewData Analyst RoleThe Data Insights Specialist plays a vital role within the organization, supporting business projects by sourcing and maintaining data pipelines.Main Responsibilities:Design and maintain data pipelines by sourcing data across departments.Develop analytical data assets for self-service insights within the team.Analyze internal and...
-
Data Pipeline Specialist
2 weeks ago
Sydney, New South Wales, Australia Bebeetechnology Full timeJob Description:Susquehanna is a global privately held financial trading firm with a unique approach to making business decisions every day.Their trading technology team performs support, deployment and automation tasks related to the management of data pipelines throughout the Asia Pacific region.Daily monitoring and market pre-open system checks are...
-
Data Pipeline Specialist
2 weeks ago
Sydney, New South Wales, Australia Bebeedata Full timeTechnical Support EngineerSig-Susquehanna is a global financial trading firm that utilizes collective wisdom and relentless problem-solving to drive business decisions.Your Role:As part of our trading technology team, you will be responsible for performing support, deployment, and automation tasks related to data pipelines in the Asia Pacific region. You...
