Expert Data Engineer

5 days ago

Sydney, New South Wales, Australia beBeeDataEngineer Full time
Seeking a seasoned data engineering professional to spearhead the development of cutting-edge data solutions.

The ideal candidate will possess extensive experience in designing and implementing large-scale data pipelines, as well as expertise in modern data engineering methodologies.

Key responsibilities include:
  • Designing and architecting scalable data solutions using Databricks Lakehouse Architecture.
  • Developing and enhancing data-driven solutions integrating with clients' systems using state-of-the-art tools.
  • Developing high-performance, batch and real-time data pipelines employing advanced streaming technologies.
  • Optimising and managing a broad array of database technologies.
  • Utilising big data frameworks to address challenges associated with large-scale data processing and analysis.
  • Implementing and overseeing cloud-specific data services.
  • Ensuring all data practices comply with security policies and regulations.
Required skills and qualifications include:
  • Databricks Platform: Hands-on experience in Databricks to build scalable, high-performance data solutions.
  • Core Data Engineering Tools & Technologies: Proficiency in SQL and Spark, familiarity with platforms such as Databricks and Snowflake.
  • Data Storage Expertise: Knowledgeable in data warehousing technologies like BigQuery, Snowflake, and Databricks.
  • Building and Managing Large-scale Data Systems: Experienced in developing and overseeing large-scale data pipelines and data-intensive applications.
  • Data Modelling Expertise: Proficient in data modelling, understanding the implications and trade-offs of various methodologies and approaches.
  • Infrastructure Configuration for Data Systems: Competent in setting up data system infrastructures using infrastructure-as-code practices.
  • Programming Languages: Proficient in Python and SQL, with additional experience in programming languages like Java, Scala, GoLang, and Rust considered advantageous.
  • CI/CD Implementation: Knowledgeable about continuous integration and continuous deployment practices using tools like GitHub Actions and ArgoCD.
  • Testing Tools and Frameworks: Experienced with data quality and testing frameworks like DBT, Great Expectations, and Soda.
  • Commercial Application of Data Engineering Expertise: Demonstrated experience in applying data engineering skills across various industries and organisations in a commercial context.
  • Consulting and Advisory Skills: Experienced in a consultancy or professional services setting, offering expert advice and crafting customised solutions that address client needs.
Professional experience and qualifications required include at least 8+ years of data engineering or equivalent experience in a commercial, enterprise, or start-up environment, consulting experience within a technology consultancy or professional services firm is highly beneficial, degree or equivalent experience in computer science or a related field.,

  • Data Engineer Consultant

    3 weeks ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time

    Join to apply for the Data Engineer Consultant role at NTT DATA, Inc.3 days ago Be among the first 25 applicantsJoin to apply for the Data Engineer Consultant role at NTT DATA, Inc.Get AI-powered advice on this job and more exclusive features.Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for...

  • Data Engineer Consultant

    3 weeks ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time

    Join to apply for the Data Engineer Consultant role at NTT DATA, Inc.3 days ago Be among the first 25 applicantsJoin to apply for the Data Engineer Consultant role at NTT DATA, Inc.Get AI-powered advice on this job and more exclusive features.Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for...

  • Data Engineer Consultant

    5 days ago

    Sydney, New South Wales, Australia NTT DATA, Inc. Full time

    Join to apply for the Data Engineer Consultant role at NTT DATA, Inc.3 days ago Be among the first 25 applicantsJoin to apply for the Data Engineer Consultant role at NTT DATA, Inc.Get AI-powered advice on this job and more exclusive features.Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for...

  • Data Engineering Expert

    3 days ago

    Sydney, New South Wales, Australia beBeeDataEngineering Full time $150,000 - $200,000

    Data Engineering ExpertThe Data Engineer Consultant role involves transforming unstructured data into a structured format that can be analyzed in a query or report.This position develops reusable and complementary data sets, analyzing data sources and structure to design and develop data models supporting business analytics requirements.Key...

  • Data Engineering Leadership Expert

    3 days ago

    Sydney, New South Wales, Australia beBeeDataEngineering Full time $180,000 - $240,000

    Job Title: Data Engineering LeadershipMain Responsibilities:Oversight of data engineering lifecycle from planning to deployment ensuring quality, reusability and alignment with platform goals.Leadership in building scalable data pipelines, models and reusable components using Snowflake and AWS-native tools.Data product development aligned to domain needs and...

  • Data Engineering Expert

    5 days ago

    Sydney, New South Wales, Australia beBeeDataEngineering Full time $120,000 - $180,000

    Data Engineering ExpertiseWe are seeking a highly skilled Data Engineer to join our growing team responsible for designing and implementing cutting-edge data engineering solutions.Our team is passionate about leveraging data to drive business value and improve customer experiences. We work closely with cross-functional teams to deliver high-quality data...

  • Senior Data Engineering Expert

    3 days ago

    Sydney, New South Wales, Australia beBeeDataEngineering Full time $120,000 - $180,000

    Job TitleA Senior Data Engineering Expert is required to lead the design and implementation of cutting-edge data platforms using modern technologies.Job DescriptionKey Responsibilities:Design and deliver innovative data solutions on cloud-based platforms, leveraging expertise in architecture, data engineering, and cloud computing.Collaborate with...

  • System Engineer

    3 weeks ago

    Sydney, New South Wales, Australia Data#3 Full time

    Join or sign in to find your next jobJoin to apply for the System Engineer role at Data#32 days ago Be among the first 25 applicantsJoin to apply for the System Engineer role at Data#3Direct message the job poster from Data#3We're looking for a technically capable Mid-Level Systems Engineer to deliver a focused, high-impact project for a key client. This is...

  • System Engineer

    3 weeks ago

    Sydney, New South Wales, Australia Data#3 Full time

    Join or sign in to find your next jobJoin to apply for the System Engineer role at Data#32 days ago Be among the first 25 applicantsJoin to apply for the System Engineer role at Data#3Direct message the job poster from Data#3We're looking for a technically capable Mid-Level Systems Engineer to deliver a focused, high-impact project for a key client. This is...

  • Data Engineering Expert

    3 days ago

    Sydney, New South Wales, Australia beBeeDataEngineering Full time $120,000 - $160,000

    Job Description">This role is focused on developing and maintaining internal software tools for data processing and workflow automation. The Senior Data Engineer will be responsible for designing, building, and refining these tools to ensure they meet the company's needs.">Key Responsibilities:">">Develop and maintain internal software tools for data...