Highly skilled Data Engineer wanted for building scalable data solutions

1 day ago

Sydney, New South Wales, Australia beBeeData Full time $120,000 - $180,000

Data Specialist

This role focuses on transforming existing proof-of-concept into scalable data solutions. Ideal candidate will have extensive knowledge of Python, especially working with PySpark.

The successful Data Engineer will be responsible for building and refining complex data pipelines to support ongoing monitoring and management. They must collaborate closely with developers and stakeholders to ensure timely delivery within a set deadline.

Key responsibilities include taking ownership of an existing PoC, building scalable data pipelines, collaborating with team members, and applying best practices in CI/CD and production deployment.

Requirements
  • Mandatory Skills:
    • Python: Advanced proficiency, particularly working with PySpark.
    • Databricks: Solid experience with Unity Catalog and Delta Live Tables.
    • RESTful APIs: Experience managing client credential requests using Python.
    • CI/CD: Deep understanding of best practices in setting up and maintaining pipelines.
    • End-to-End Development: Confidence in pushing solutions into production environments.
  • Nice-to-Have Skills:
    • SQL: Useful for data manipulation and validation.
    • Microsoft Graph API: Familiarity is beneficial but not required.

  • Data Architect and Engineer

    5 days ago

    Sydney, New South Wales, Australia beBeeDataEngineering Full time $180,000 - $200,000

    Job Title: Data Operations EngineerWe're seeking a skilled and experienced data operations engineer to join our team. As a key member of our integration platform, you will be responsible for designing, building, and maintaining scalable and robust data platforms.Key Responsibilities:Design and build data architectures that meet business requirementsMaintain...

  • Building Scalable Data Solutions

    7 days ago

    Sydney, New South Wales, Australia beBeeData Full time $150,000 - $180,000

    Data Solutions EngineerWe are committed to building scalable data solutions by leveraging cloud infrastructure and collaborating with global teams.At our organization, diverse professionals come together to drive innovation in 34 markets.Key ResponsibilitiesDesign, develop, deploy, and maintain data assets on Google Cloud Platform (GCP).Coordinate project...

  • Highly Skilled Data Engineer Wanted

    1 day ago

    Sydney, New South Wales, Australia beBeeDataEngineer Full time $120,000 - $180,000

    Job OverviewWe are seeking a highly skilled data engineer to join our growing team.In this role, you will design, build, and optimize robust data pipelines and modern analytics solutions using cutting-edge technologies.Key Responsibilities:Design, develop, and maintain scalable data pipelines and ETL/ELT processes in Azure.Develop and maintain data models,...

  • Building Scalable Data Solutions

    3 days ago

    Sydney, New South Wales, Australia beBeeData Full time $180,000 - $220,000

    Job Title: Enterprise Data ArchitectCuscal is shaping the future of payments and pioneering innovative solutions that millions of Australians rely on every day. With over 59 years of experience, we're preparing for what's next.As an Enterprise Data Architect in our Data Governance and Data Platform team, you'll develop, maintain, and support ground-breaking...

  • Data Engineer Consultant

    3 weeks ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time

    Join to apply for the Data Engineer Consultant role at NTT DATA, Inc.3 days ago Be among the first 25 applicantsJoin to apply for the Data Engineer Consultant role at NTT DATA, Inc.Get AI-powered advice on this job and more exclusive features.Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for...

  • Data Engineer Consultant

    3 weeks ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time

    Join to apply for the Data Engineer Consultant role at NTT DATA, Inc.3 days ago Be among the first 25 applicantsJoin to apply for the Data Engineer Consultant role at NTT DATA, Inc.Get AI-powered advice on this job and more exclusive features.Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for...

  • Data Engineer Consultant

    6 days ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time

    Join to apply for the Data Engineer Consultant role at NTT DATA, Inc.3 days ago Be among the first 25 applicantsJoin to apply for the Data Engineer Consultant role at NTT DATA, Inc.Get AI-powered advice on this job and more exclusive features.Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for...

  • Building Scalable Data Pipelines

    2 days ago

    Sydney, New South Wales, Australia beBeeDataEngineer Full time $120,000 - $180,000

    Job Title: Data EngineerWe are seeking a skilled Data Engineer to join our organization.Data Engineers will be responsible for designing, building, and maintaining scalable data pipelines and workflows on the Databricks platform.Main Responsibilities:Collaborate with data engineers, data scientists, and analysts to optimize data architecture and...

  • Highly Skilled Data Professional

    2 hours ago

    Sydney, New South Wales, Australia beBeeDataEngineer Full time $100,000 - $150,000

    Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team.Design, build, and maintain scalable data pipelines using Azure Data Factory for data ingestion from diverse sources.Utilize Azure Data Lake Storage (Gen2) and Delta Lake for managing data in different stages.Implement data processing workflows using Azure Databricks to...

  • Highly Skilled Data Engineer Wanted

    19 hours ago

    Sydney, New South Wales, Australia beBeeData Full time $120,000 - $160,000

    Senior Data Engineer PositionThis role is a challenging opportunity for a highly skilled data engineer to lead on medium to complex products and product development. The successful candidate will apply advanced technical knowledge of data engineering principles and practices, coupled with specific data technology, domain expertise and thought leadership.The...