Key Player in Data Pipeline Development

19 hours ago

Sydney, New South Wales, Australia beBeeData Full time $140,000 - $170,000

Job Title: Data Engineer

">
  • We are seeking a skilled professional to design, develop and implement data pipelines from both internal and external sources.
  • The successful candidate will have experience with data transformation and modeling, combining complex datasets for downstream consumption and visualization.
  • They will work primarily with SQL and Python to automate and optimize data loading and transformation systems.
  • Maintain, operate, and tune Snowflake and DBT.
  • Work with the AI team to develop and automate processes using RAG technologies.
  • Monitor and troubleshoot operational or data issues in the data pipelines.
  • Drive architectural plans and implementation for future data storage, reporting, and analytic solutions.
  • Design, develop, and maintain robust ETL pipelines to support data ingestion, transformation, and integration from various sources.
  • Write complex and optimized SQL queries for data extraction, analysis, and transformation across large datasets.
  • Manage and optimize data storage solutions, including data warehouses (e.g., Snowflake, Redshift).
  • Troubleshoot ETL failures and performance issues in SQL scripts or data processes.
Required Skills and Qualifications

The ideal candidate will have demonstrable experience of working in a complex Data Science environment, applying Machine Learning techniques, advanced analytics, and statistical modeling.

  • Experience with Kafka
  • Strong knowledge of AWS cloud-native infrastructure
  • Proficiency with Python Scripting
  • Strong Experience with AWS RDS Query Optimisation (Aurora Postgres and Aurora MySQL)
  • Experience with workflow management engines (i.e. Airflow, AWS Step Functions)
  • Experience working with Snowflake and DBT

You will be a motivated self-starter who enjoys dealing with a myriad of internal stakeholders and will be able to coach senior executives along the data science journey.

  • A Key Player in Data Management

    7 days ago

    Sydney, New South Wales, Australia beBeeData Full time $100,000 - $130,000

    Job TitleSenior AnalystA key player in ensuring data quality, integrity and governance across the organisation. This role fosters collaboration with stakeholders to drive best practices in data management and compliance.

  • A Key Player In Data Management

    9 hours ago

    Sydney, New South Wales, Australia Bebeedata Full time

    Job TitleSenior AnalystA key player in ensuring data quality, integrity and governance across the organisation. This role fosters collaboration with stakeholders to drive best practices in data management and compliance.

  • Lead Data Pipeline Developer

    15 hours ago

    Sydney, New South Wales, Australia beBeeDataEngineering Full time $120,000 - $160,000

    Software Engineer PositionWe are seeking an experienced software engineer to collaborate with our global teams and contribute to the development of cutting-edge software solutions that support large-scale data processing and analytics.Job Description:We design and build custom software components for data pipelines using Python or other programming...

  • Data Pipeline Specialist

    3 days ago

    Sydney, New South Wales, Australia beBeePipeline Full time $140,000 - $180,000

    Job Opportunity: Data Pipeline SpecialistWe are seeking a highly skilled Data Pipeline Specialist to join our team. The successful candidate will be responsible for designing, implementing, and maintaining robust data pipelines that integrate data from diverse sources.The ideal candidate will have strong proficiency in SQL and Python for data manipulation,...

  • Senior Data Pipeline Developer

    24 hours ago

    Sydney, New South Wales, Australia beBeeData Full time $150,000 - $200,000

    Job Description:As a seasoned professional, you have the opportunity to play a pivotal role in our Data Platforms team. The ideal candidate will be responsible for designing and building scalable data pipelines that simplify our technology landscape and deliver measurable business outcomes.Responsibilities:Designing and developing high-quality data...

  • Data Pipeline Specialist

    2 weeks ago

    Sydney, New South Wales, Australia beBeeDataEngineer Full time

    About the RoleDrive's Data Operations team is seeking an ETL Engineer to help us achieve our vision of shaping the future of automotive in Australia. As a key member of this team, you will be responsible for designing, implementing, and maintaining efficient data pipelines that drive business value.The ideal candidate will have 5+ years of experience with...

  • Data Pipeline Specialist

    7 days ago

    Sydney, New South Wales, Australia beBeeDevopser Full time $150,000 - $170,000

    Job DescriptionAs a DevOps Engineer, you will play a vital role in our trading technology team. You will be responsible for performing daily monitoring and market pre-open system checks utilizing automation, developing new tools and utilities to help build data pipelines and studies, and becoming a subject matter expert in our in-house systems and data...

  • Data Pipeline Specialist

    1 day ago

    Sydney, New South Wales, Australia Bebeeautomation Full time

    Job Title: Devops EngineerAbout the Job:Susquehanna is a global privately held financial trading firm. Our founders are entrepreneurs with a drive to deeply understand the world around us and use that understanding to make the best business decisions every day.As part of our trading technology team, you will perform support, deployment, and automate tasks...

  • Data Pipeline Architect

    3 days ago

    Sydney, New South Wales, Australia beBeeData Full time $100,000 - $170,000

    Job DescriptionAs a Senior Data Engineer, you will play a pivotal role in crafting and implementing cutting-edge data pipelines that transform raw market data into actionable insights for traders worldwide.Our ideal candidate will possess exceptional technical skills, paired with outstanding collaboration and problem-solving abilities.Key...

  • Data Pipeline Specialist

    3 days ago

    Sydney, New South Wales, Australia beBeeAutomation Full time $120,000 - $180,000

    Job Title: Devops EngineerAbout the Job:Susquehanna is a global privately held financial trading firm. Our founders are entrepreneurs with a drive to deeply understand the world around us and use that understanding to make the best business decisions every day.As part of our trading technology team, you will perform support, deployment, and automate tasks...